President’s column: Black box solution

If anyone read the article written by Hugo Rifkind entitled ‘Horizon was ‘computer says no’ on a vast scale’, which was published in The Times newspaper in January of this year, or indeed you remember my inaugural NSC column as Deputy President back in December 2022, then please accept my apologies because you will already be familiar with the concerns and arguments that I feel the need to reiterate below.

The recent Post Office scandal, where many staff were wrongly accused of malpractice and theft has now been proven to have been the result of a computer programme fault that was not adequately monitored or managed by humans. This, of course, currently ignores any suggestion that distant parties may have had remote access to every computer terminal, which would point to a very different human led crime.

The software, named Horizon, was developed and released by the very reputable and prominent company Fujitsu, and The Times article goes on to say that: “Horizon is an extreme example but not as an isolated problem.” I find this somewhat concerning, as we start to rely on software solutions to many of our engineering challenges, including both simple and complex steelwork connection designs.

In our daily lives, when dealing with domestic issues, how many of us find ourselves listening to a computer-generated voice asking us to choose between option one to five, or listening to a pre-recorded message telling us that we can manage our issue on their user-friendly website or downloaded App?

There are increasing examples of individuals being asked to pay huge sums of money for implausible changes to household bills that are incomprehensible, other than the fact that the computer says ‘yes’. And with no-one on the end of the phone with whom to discuss a logical solution for this situation.

Which leads me to my previous (Dec 2022) topic of over-reliance on computers.

The Post Office/Horizon issue happened in a completely different industry to ours, but does it create the same potential challenges? In their case, it related to people’s livelihood and morality and in our case, it relates to competence and public safety. Some might argue that there is a measurable difference when it comes to safety, but I’m sure that those involved in the Post Office scandal would see it differently.

With the next generation of Eurocodes due to be published soon, in my opinion, it will become increasingly difficult to figure out the fact from the fiction, since many of the proposed design approaches and corresponding formulae are generated from academic studies rather than full-scale tests and historical experience. This will undoubtedly result in further software and AI releases and while there is no evidence to say that these approaches will be unsafe, they will be extremely difficult to measure and verify by the more traditional and conservative methods.

We need to be openly aware of the challenges that lie ahead with the introduction of AI and the increase in ‘black-box’ software solutions. Our current design solutions may indeed be conservative, but the many years of experience clearly demonstrate that they are both safe and reliable.

So, looking beyond the horizon, is it not time to introduce ‘Computer-Free-Fridays’ to engage our future engineers in the hands-on experience that is essential for proven, practical and safe design solutions?

Gary Simmons
BCSA President