Monday, March 20, 2023

It Is Never a Ten Minute Fix


Being a technology professional is filled with ups and downs. It is a well-paid profession, but it comes with plenty of trade-offs. I can cite many examples. I remember getting a tech-support call when celebrating my anniversary dinner with my ex-spouse. One time, I was referred to as the "nerd boy" in the office by the Vice President of marketing. My company fired me a week before Christmas because I made a mistake after working sixteen hours the night before to fix a problem with the credit card billing system. The compensation is excellent, but the job security is harsh, considering that business people think what we do is easy. It explains why software professionals are cranky because each change could introduce a catastrophic failure that can cost people their careers. To save a few hair follicles for business and technology professionals, I am talking about why a quick change is anything but fast. 

I cannot speak for others, but my training in agile has taught me that I should be responsive to change as a technology professional. Plans and the business world transform with frighting regularity, so it makes sense that an intelligent business professional can pivot from their current situation to a different one. What drives technology professionals crazy are what we call ego-ware requests. An ego-ware request is a change made to a system that delivers questionable value but satisfies the ego of the person making the request. For example, a UX designer demanded a text box on a mobile web page move a tenth of a pixel because it created more harmony on the page. Such a minor change took three developers at a rate of $75 an hour three weeks to figure out, which cost the business $18,000. The money did not have any impact on the revenue of the firm. It only achieved one thing: the designer's ego gratification. The now infamous Superbowl meltdown of Elon Musk is also a classic example of ego-ware taken to the extreme. 

Besides the ego-ware request, you often get a "quick change" from colleagues who mean well but do not understand the ramifications of that request on the overall systems. I remember someone asking me to add a field to an HTML web form and make it optional so the call center could use it for notes. The call center supervisor said, "Come on, Ed, it should only take ten minutes." I firmly believe that statement is a sign pointing toward hell. 

Long story short, I made the change. I did some testing and then deployed it to UAT. I was a hero in the afternoon. Three weeks later, I was fired because the change created production problems, preventing customers from making payments. The extra field broke the payment API, and customers were ticked off. The development manager looks at source control and the last production push with my name on it. My boss told me to pack my desk and go home. Stories like this are familiar in the technology business, and it is why developers do not like to make arbitrary changes. 

What looks like a ten-minute fix to the business is a series of events requiring time and attention to detail. The actual code change will take ten minutes. Updating all the unit tests for the application might take four hours. Quality assurance can finish testing in a day if they are not busy. After testing, it will be deployed to a UAT environment so the business can test deployment to production. If anything goes wrong, the entire process starts over again. A ten-minute fix is a whole week's worth of work. It is another reason software engineers get cranky because the change may not be significant, but the implications on the system are. 

To recap, software engineers, are grumpy about system changes because it could first put their careers at risk if something goes wrong. Murphy's Law impacts technology more than any other part of the business, so the risk is higher than most professionals will accept. Second, to avoid those risks, the organization creates test-driven development, quality assurance people, and stage gates to software release. The deliberate friction between the idea and the actual costs, time, money, and the frayed nerves of the IT staff means that if you change a system, you better have a damn good reason. 

Software is one of the few human professions that can not be automated. So the next time you ask a technology person to make a minor change and they tell you to jump into the lake, the software engineer is not being insubordinate; they are responding rationally to something which might threaten their career. It takes real people time to make each technology dream a reality. 

Until next time. 


No comments:

Post a Comment