You don’t need me to convince you of the dangers of AI at a macro level, with everyone from Elon Musk to Maureen Dowd calling for a pause on the technology. But none of it matters.  Because A16Z is talking about the coming AI-augmented 1000x developer, and others claim AI coding is becoming so powerful humans will stop writing code in the next 10 years. Sequoia (one of Bloomfilter’s investors) has mapped the rise of AI across the SDLC, while Microsoft keeps investing in Co-Pilot. Unless there’s massive government prohibition (and we know how well prohibition works)—AI will change the nature of software development. Software is eating the world, and AI is eating software.

In a world where robots write software, understanding the software development process is more important than ever. Not in the sense that we need to protect the world from the Singularity or Skynet, but to ensure your company’s software development aligns with the business, as AI changes the way alignment happens.

Executives are always telling me software development is a “black box.” It’s too technical. Too hard to understand what’s being built. Why are certain features or capabilities prioritized? Why are various tradeoffs being made in functionality? How do enhancements impact performance, scalability, or security?

To answer these questions, we need transparency in the process. But if artificial neural networks generate outputs, we may not understand how or why they came to the decision they did. LLMs will confidently answer math questions incorrectly and they can’t show their work when asked to explain their answers. They may even become hostile or abusive.

If you think it's tough to understand the SDLC right now, imagine not even being able to call a meeting to discuss what's being built and why.

We know AI will accelerate the total lines of code generated within a company. That’s great if you measure development by arbitrary (and flawed) metrics around developer productivity. Unfortunately, lines of code don’t just need to be written, they also need to be maintained and profiled for security vulnerabilities. They are a liability to the company, which is why we talk about “technical debt.”

More code = more technical debt. That risk may be worth taking if the development aligns with the company strategy, is QA’d for efficacy, and maintained intelligently. But we can’t skip those steps. Writing code is just one step of the SDLC, and speeding it up doesn’t guarantee better business outcomes.

To harness the coming AI revolution and solve other software development challenges, organizations must treat software development as a business process. Like any process, it has inputs, outputs, and constraints. It can be inscrutable to outside observers. By understanding and instrumenting the entire process, we can gain a better understanding of what’s happening within it.

In the past, automation performed a single task repeatedly. In that world, it was reasonably straightforward to introduce quality assurance into the process to ensure the output matched specification. But no two software development projects are ever the same. As we consider automating software development, it’s critical we have the ability to understand what’s happening inside the process.

Achieving this understanding requires process mining. Process mining lets us peer inside a complex business process and analyze the metadata of core source systems. With this view, we can understand why certain decisions are made, why errors occur, and where inefficiencies reduce output. By leveraging process mining, organizations can ensure their software development stays better aligned with the business and is delivered more efficiently.

Software development is a business process, and like any other business process it has the potential to be accelerated and automated. As we accelerate the process, we must not give up visibility into what’s happening. Consider how you’ll ensure alignment with business objectives. Without intelligence and visibility into the process, AI-assisted development will simply be a flawed process done faster, resulting in worse outcomes for the business.

I’ll let others opine about what the future of AI means for humanity and how to prepare for it. If we want to be ready for the impact of AI on software development, we need visibility into the software development process. If we don’t understand how our software development works now, we can’t hope to stay on top of it as the velocity accelerates and machines make questioning the process even more difficult. It may not be quite as terrifying as SkyNet, but the idea of even worse software outcomes for businesses is plenty frightening in its own right.