17 Jul 2022
I have seen a lot of articles lately that have the take, “GitHub Copilot is going to replace all Software Engineers” or “Why become a Software Engineer now that AI can do your job?” I can understand how the layperson may hear a description of GitHub Copilot or see a demo and think that the AI must be knowledgeable, but to those who understand the job of Software Engineer, we can quickly see the benefits and the limitations of AI written code.
Why GitHub Copilot can’t replace Software Engineers right now
In this section, I’m going to outline the reasons that the idea that GitHub Copilot and similar tools are going to replace all Software Engineers and
Understanding the problem you are trying to solve
There is an old joke “You can never replace Software Engineers with AI because it would first require the business stakeholder to be able to articulate what they want.” In my career as a Software Engineer, this was always the biggest challenge we faced. It has not been unusual to get only a vague sentence or two to describe complex behavior or worse, “I’ll know it when I see it.”. Additionally, I have often received requirements counter to previous requirements and must work to try and develop a solution within those constraints. These types of requirements are challenging to communicate to even the most powerful AI currently available.
The first role of Software Engineers is not to code but to solve problems
I volunteer teaching High School Programming, and one of my first lessons is that a Software Engineer’s job is to solve problems, and sometimes those problems may be best solved by writing NO code at all. I was once hired as a consultant for a company looking to build a pretty expensive software solution. Over the next couple of hours of us discussing the requirements, it became very apparent that we could eliminate the entire project with a few minor changes to their process. While it meant I didn’t get to bill for the software project, it was the right thing for them, and I valued providing the BEST solution over the most profitable one.
As it stands, tools like GitHub Copilot don’t have that additional context and can’t suggest those changes. It’s a very skilled hammer and sees every problem as a nail.
Experience has shown that GitHub Copilot can introduce subtle bugs
I’ve been using GitHub Copilot since the early betas, and I’ve been highly impressed with the suggestions that it makes but often will suggest baffling code snippets that include extraneous logic and occasionally ones that compile but are wrong in some subtle way. You have to understand the code it generates to catch some of these bugs on the first pass, and I’ve watched several people happily accept the suggestion and move on, never noticing that it just broke the logic of the code.
Some examples of things that it might suggest but could be wrong:
- Properties with the wrong types.
- Conditionals with the wrong condition logic.
- Extra properties that aren’t needed.
- Classes and/or methods with the wrong scope.
- Methods that require external frameworks that you don’t use.
- Outdated syntax.
GitHub Copilot is only as good as the code the models were trained on
Theodore Sturgeon once observed of contemporary Science Fiction, “Ninety percent of everything is crap.” This has often been applied to other areas, including software. While I think 90% is a high number, there is a lot of bad code in the wild and much of it was used when training the models. That is not to say that it will always output bad code, but sometimes it will. The model has developed connections that will, in many cases suggest the best code, but just as often it will suggest code that can be either bad or at the very least “smell.”
It takes a seasoned Software Engineer to spot the difference. If you blindly accept every suggestion, you will almost certainly end up with sub-par code.
All that said, you should give GitHub Copilot a try
I’ve spent the bulk of this article highlighting the weaknesses of AI-driven tools like GitHub Copilot, but now I’d like to dive into why I think Software Engineers SHOULD use it.
It automates a way a lot of mundane, repetitive coding
Programming is not all complex logic and clever tricks. It’s a lot of coding, very simple structures, and writing mundane code. GitHub Copilot excels at this. Create a new class and start writing the first property, and will often suggest the rest. It’s not perfect as stated above, but it’s surprisingly accurate.
It helps you remember how to do things that you don’t do very often
There have been many times that I needed to perform an operation or utilize a method. I can’t remember the method’s name or exactly how to do something in a language and where I would have previously “Googled” it or copied and pasted from StackOverflow. I was pleasantly surprised it suggested just what I needed to do. It allowed me to stay in the moment and not break flow.
I find it extremely helpful with determinant “assert statements” in unit tests
One area that I have found GitHub Copilot extremely useful is in writing the “assert statements” in unit tests. As soon as I start writing one, it will suggest additional asserts, and they have frequently been asserts I had not considered. It’s like having a very perceptive coding partner.
Its suggestions can help spur ideas when you get stuck
Sometimes you get stuck writing a particular piece of code and GitHub Copilot can often offer suggestions that either solve a problem or at least help point you in the right direction.
GitHub Copilot is a tool and can be very helpful in increasing your productivity but it’s not a panacea. Think of it more like “Cruise Control” in your car which helps by taking over holding your speed on long stretches, but it’s not “auto-pilot.” You hold the course and it assists. As a Software Engineer, you still need to understand the problem and determine the solution but the tools can help you get to your destination with less effort.
Finally, do I think there will ever be a tool that could replace a programmer? Yes, on a long enough time scale I think it’s almost certain. The evidence I’ve seen points to that being a ways off and may require new approaches to AI and Machine Learning, but who knows, we could be just one breakthrough away. When that time arrives, we will know it, but as of today, good Software Engineers are needed more than ever!
20 Nov 2019
Recently, a coworker and I were discussing “Inner-sourcing.” Inner-sourcing is when a company utilizes open source principles and practices within the company. One thing we were lamenting was that even as our company works to implement Inner-sourcing, repos are still tied to teams and thus their bandwidth is a limiting factor.
This got me thinking, how could we setup a git repo which was not owned by any one person or team but still had the appropriate gates to prevent bad code from entering? That’s when I had the idea for a new ownership model, “Meritocratic Ownership.”
In this model, credit (for a lack of better terms, we’ll call it ‘Karma’) is awarded to individuals as they contribute. Over time, as their “Karma” increases they have more sway over a repo that has been configured with the “Meritocratic Ownership” model. However, the real power, is pooling Karma to create or accept change as a group. In the next section I will outline at a high-level how I envision this working.
How does a repo become “meritocratic?”
This model does not replace any of the previous ownership models. This would be something that the original repo owner would have to choose to do. However, once it’s been turned on, it could only be turned off by a community vote.
What is Karma and how do I accrue it?
Karma, is a credit that is earned when you perform actions that promote the healthy growth of a repo or the community at large.I see three different types of Karma:
Repo Karma – This is the highest form. It is awarded as you submit pull requests that are accepted, write tests, improve documentation or other things necessary for a healthy repo.
Community Karma – This is awarded as a percentage of your repo Karma. The more you contribute to open source projects, the more Karma you earn and this can be used to give an individual gravitas with newer repos since they have proven their merit. However, this Karma is not as valuable as repo Karma.
Awarded Karma – This is a limited supply that can be distributed from one individual to another as a “thank-you” for work. It essentially acts as an incentive to enlist help.
How is Karma used?
When something happens on a repo that needs approval it is presented to the community. Each action requires a certain amount of consensus in order to be acted upon. For example, accepting a pull request may require 200 Karma for a particular repo. If 4 people who each have 50 Karma review and approve it, the pull request is accepted. Alternatively one person who has sufficient Karma can accept it. This can also be challenged by other contributors in order to keep balance.Another important point is the amount of Karma required for a particular action is based on a formula that is designed to ensure the largest amount of community involvement, similar to Bitcoin.
While I originally envisioned it for use internally, it would be great for community open source projects as well as reviving once-dead projects. While you can always fork a dead project and revive it that way it is also diluting recognition and raising the barrier to entry.
This proposal is by no means perfect and I highly encourage you to leave a comment with suggestions or tweet at me @helmsb.
20 Nov 2019
You may be asking yourself right now, “What is Good Will Equity?” The simple definition is that Good Will Equity is an imaginary account that everyone has internally that is either deposited into or withdrawn from when asked to do things by their employer.
A deposit is made when the company does something good. For example , giving you a raise, giving you a promotion, having a particularly good year in profit sharing distributions, etc.
A withdrawal is made when you ask an employee to do things that they don’t like. For example, asking them to work late or on the weekends, asking them to travel unexpectedly, etc.
Why the financial analogy?
I’ve always thought that this concept ties in well with the financial analogy as the parallels are numerous. Just like a bank account you can spend more than you put in. Eventually, you go negative and you have to change your habits like or not, or the bank may decide your not worth it and terminate your account. In a similar fashion a team-member may decide that what your asking is too much and decide to leave and go somewhere else.
You can also rack up debt on a credit card constantly making withdrawals and giving shallow deposits in the form of meaningless gestures, fluffy messages, etc. It may put off the inevitable for a while but do it long enough and eventually you don’t have the Good Will reserves to do the important things (ie. You can’t pay the mortgage).
Finally, in keeping with the financial analogy, the way you treat your team-members can have further reaching repercussions. Just like a bad credit score due to missed payments can keep you from being approved for a mortgage, similarly, burning out your team can cause you to not only lose them but to have the well poisoned for future candidates. You may suddenly find yourself in a position where it is difficult to hire because no one trusts you anymore.
Why is this important to me?
It is very important at that as leaders we never lose sight of this “Good Will Equity” bucket. It is easily to make decisions purely on the basis of monetary needs or gut-feeling but if we don’t take into account the “Good Will Equity” implications we can quickly lose our most talented individuals over things that honestly just don’t matter.
How can we know what a particular person’s “Good Will Equity” balance is?
Unfortunately there isn’t a website we can visit or a phone number to call to check the current balance. You have to build a relationship with your team members and listen to them to find this out. This can be through one-on-ones, causal conversations or open discussing in things like team retros.
What Can I do with this information?
As you gain an understanding you have to stand up as be an advocate for your team and make sure that the company leadership understands the more intangible costs of a decision. You must ensure that when you ask a team-member to do something that you know costs “Good Will Equity” that you are doing it for the right reasons. You must also offset these withdrawals with deposits and insurer that you are showing your team the appreciation for the value that they provide in meaningful ways.
In summary, every decision that you make that affects your team doesn’t have only monetary costs but also has more personal intangible costs and as a strong leader you need to be cognizant of these costs and be an advocate for your team to make sure that when we are spending “Good Will Equity” it’s for the right reasons and that we are constantly paying back in.
20 Nov 2019
Gene Kranz was a NASA Flight Director who served from the first days of the Mercury missions all the way into the Space Shuttle era. He was immortalized in the cinematic depiction of the real life events of “Apollo 13.” In it, Kranz, played by Ed Harris utters one of the most memorable lines in the movie, “Failure is NOT an option” and when the stakes are the lives of the 3 men in the badly damaged command module I can certainly not disagree. However, for many of us the stakes are much lower. Most of the software we write is not tied to orbiting space craft, nuclear warheads or medical life support hardware 2 and thus failure IS and SHOULD be an option.
I’ve been in software development professionally for over 15 years. During this time a lot has changed in how we write software. I’m not just talking about the tools and the languages but the methodologies as well. When I began my career we were very much a “Waterfall” based shop. Over time we tweaked that methodology but fundamentally it remained the same. Then, like a great lightning bolt from the sky came “Agile.” It was going to solve all of our problems. Never again would software be late or requirements go unfulfilled. All we had to do works work “Agilely” by doing a very specific set of pre-determined ceremonies and actions and like magic all would be right in the world. Needless to say, that didn’t happen.
In many cases things were worse. The requirements were less defined so our customers were upset. We planned all the work carefully and then worked feverishly all day, most nights and nearly every weekends in order to fulfill the plan that was unrealistic when we devised it.
What went wrong? Wasn’t “Agile” supposed to save us from all this?
In the previous paragraphs, I capitalized “Agile” to differentiate it from true “agile software development.” “Agile” has gone from an idea, to a buzz word, to a brand. All along the way it has picked up new cruft, like a snowball rolling down a hill. Consultants have come and gone, companies have risen and fallen in the search to package, promote and sell “Agile” to a waiting public. The problem began almost immediately with a misconception and a mangling of the concept. The word “agile” as layed out in the “Agile Manifesto” is an adjective not a noun. This might seem like an unimportant distinction but it’s the cause of decades of strife.
Agile software development doesn’t dictate whether you use Kanban or Scrum or what time you need to have a “Daily Stand-up” or if you should have one at all! Instead it gives you the permission to measure things and determine if it’s effective for you and your team. This is the key point I hope to make in this post.
There is no one size fits all solution. To truly work agilely, you need to develop your practices by measuring them against the principals you want to live by and have an open and honest discussion within your team about whether or not what you’re doing is effective based on the success criteria you’ve decided upon.
Crucially, however, you MUST be willing to admit when an experiment was a failure, learn from it and develop a new experiment based on those findings.
There can be a number of challenges that can hinder this mindset. Let’s discuss a few:
The Sunk Cost Fallacy
This concept is basically that when evaluating whether or not we should continue a practice we are often biased toward continuing due to the high costs that have already been incurred. It can also serve as a way to “defer” admitting failure by assuming that since the full cost hasn’t yet been realized failure has not yet occurred. Instead you should always base your decisions on the actual data and be willing to accept your loss and work toward a better solution
Top Down Management
In many cases the culture of the company can create a situation where individual teams don’t have the necessary autonomy to make the decisions on their own and instead are forced to stay within in a strict framework of operation 3.
In some cases you may be succeeding in spite of bad practices. Former President of Pixar, Ed Catmul once said “Success hides problems.” This may be the case in your organization. When things are going well it’s difficult to look introspectively and create experiments. No one wants to rock the boat. You must fight this instinct and always be willing to admit that there are ways that you can improve.
You can only have real change in an organization of the members of that organization care about what they are doing and are looking for ways to improve and grow. This is not only those at the top but also your fellow team-mates. Everyone has to be onboard with improving which means they also have to be onboard with admitting failure.
Which brings us to probably the hardest one of all. Sometimes the one person that it is most difficult to admit failure to is yourself. The very though of failing at something makes you sick but failure is how we learn and grow and admitting to that failure is the first step in unleashing that power. You have to stop tying your self worth to the actions you perform. A failure of an experiment doesn’t define you instead it is the first step in helping you improve.
How do we begin?
First, determine an area that you are looking to measure and improve. Don’t try and solve every problem all at once. Pick something isolated and small. Begin by breaking down the problem using techniques like:
- The Five Why’s
- Mind Maps
- A3 Analysis
Note: I’ll be going more in-depth into these techniques in future posts.
Next, determine which metrics you are going to use to measure success or failure. Be realistic. Don’t set unattainable goals but don’t sand bag and set goals that are too easy either. These metrics should also correspond to real indicators that give a sense of whether or not you’ve improved the situation and not simply an easy way to “game the system” (ie. “Don’t juke the stats”). An example of this might be meeting the criteria of “reduce errors by 10%” by simply removing the error logging code.
Once you’ve determined what you want to improve, what metrics are important and how you will measure success or failure you need to determine a time-box to run your experiment then put it into practice.
The final step which the purpose of this post is to review your metrics and determine whether or not you have been successful. If you were, then great, on to the next issue. If not, then you need to have an open and honest discussion with your team and the stakeholders and review what you’ve learned and use these findings to drive the next experiment.
This cycle is sometimes called PDSA (Plan, Do, Study, Act) and it’s probably the most important practice you can add to your organization to drive a culture of agile development. Basically you should never rest on your laurels but instead constantly be experimenting and looking for ways to improve.
Positive change can only happen when you decide to take a hard look at what you’re doing and having the courage to admit failure.
Failure IS an option!
20 Nov 2019
Wilt Chamberlain was the greatest basketball player who ever lived. His most notable achievement (in basketball at least) was in 1962, while playing for the Philadelphia Warriors he scored 100 points against the New York Knicks, a record that still stands today. Even more impressive that night was that Chamberlain, a player known for being a terrible free throw shooter, made 28 out of 32 free throws. The story of this game was recently discussed on the excellent “Revisionist History” podcast. In the episode, Malcolm Gladwell recounts how Chamberlain had switched free throw techniques and began shooting his free throws underhanded (sometimes known as a “Granny Shot“). Suddenly, Chamberlain went from a terrible free throw shooter to one of the best. He was now UNSTOPPABLE!
Wilt Chamberlain shooting a free throw underhanded. AKA The “Granny Shot” with one team mate and one defender.
Wilt Chamberlain shooting a free throw underhanded. AKA The “Granny Shot”.
Except…not long after, he switched back to his old style of free throw shooting and his completion ratio tanked. He went from shooting 87.5% shooting underhanded back to shooting 40% overhanded. His only achilles heel had been his inability to shoot free throws and as soon as he’d mastered them he went back to the old way. Why would he do that? Well, Chamberlain put it this way in his biography, “I felt silly, like a sissy, shooting underhanded. I know I was wrong. I know some of the best foul shooters in history shot that way. Even now, the best one in the NBA, Rick Barry, shoots underhanded. I just couldn’t do it.” Even though all the statistics proved that shooting underhanded was the best way to go and even after he had seen the success first hand in his record breaking game he still chose to ignore it. On the other hand Rick Barry, the greatest free thrower of all time understood the advantage of underhanded free throws and embraced it to great success.
I felt silly, like a sissy, shooting underhanded. I know I was wrong. I know some of the best foul shooters in history shot that way. Even now, the best one in the NBA, Rick Barry, shoots underhanded. I just couldn’t do it.
This may seem unthinkable to many people but as software developers many of us do the exact same thing. I’m talking about creating and maintaining a culture of “Sustainable Code.”
What do I mean by “Sustainable Code?” Sustainable Code is a methodology that intends to build software using proven techniques to ensure a lower defect rate, faster turn around time on features, shorter regression test period and better work life balance for the team.
How is this achieved? There are a couple of core disciplines that over the years I’ve learned can make a huge difference when it comes to building a culture that writes sustainable code, I will be going into these in more detail in future posts.
These include unit testing, integration testing, automated feature testing.
Pair and Squad Programming
Working with another developer on a single development task or even having the entire team in a room working on a single ticket.
Sharing your work with the team and getting feedback.
Don’t get bogged down for weeks at a time working on a single feature. Write code, check-in and deploy often.
Close collaboration between devs, QA and business analysts.
We’re all one team, don’t just throw things over the wall.
Now, if you’ve been a developer any length of time you’ve probably run into every item in the list but based on statistics you most likely practice few if any of them. One study showed that TDD lowered defect rates between 40% and 90% when compared to non-TDD projects while only increasing development time 15%-35%. Even in the worst case scenario this is a net-gain! However, just like in the case of Wilt Chamberlain, most developers chose to ignore the stats and keep doing what they’re doing. Most developers balk at having someone review their code or think unit testing is a waste of time and don’t think they need it.
I’ll be the first to say that early on I HATED writing unit tests. I thought they were so pointless and just took time a way from “real development.” I didn’t want to do code reviews because in my mind I was a good programmer and I didn’t want someone else telling me how to do my job. But as I matured as a developer I came to embrace testing. It lowered my stress level when doing a refactor because I always knew I had the tests to rely on to tell me if I’d done something wrong. The same goes for code reviews. I now look forward to them because its a way to learn new ideas and gain new perspective. I get great feedback and the entire team has a better understanding of what each person is working on.
Most importantly though as our organization has adopted agile practices, I’ve realized that without these things we could not really say that we’re practicing lean. Gone are the days of multi-week regression cycles, hours long deployments, huge backlogs of defects.
As a developer. don’t be like Wilt Chamberlain and ignore proven practices because of pride or other reasons. Be like Rick Barry and use these practices to your advantage.