Source: Dynatrace

Shifting Left

At its core, Shifting Left is about reversing the conventional sequence of development tasks. Instead of addressing issues during the latter stages of development or even after deployment, this approach encourages tackling challenges much earlier in the process. Initially, this was performing functionality tests, the primary objective to catch and mitigate problems as close to their origin as possible and reducing the potential for costly and time-consuming fixes later down the line. However, today, developers are expected to be well-versed in a variety of domains including web frameworks, architectural patterns, testing approaches, build systems, diverse database types, caching mechanisms, automation utilities, container orchestration, networking concepts spanning from L4 to L7, SaaS APIs, monitoring solutions, a multitude of public clouds, and perhaps even possess some foundational knowledge in machine learning. It’s too much!

Key Principles

Below are some of the main principles of what shifting left is about. They're things you should look at implementing to achieve success in this area with the aim to "fail fast", where issues are identified and addressed quickly rather than allowing them to escalate. This minimises the impact of defects on the overall project. Ideally, developers should be able to perform these things on their laptop.

Early Testing and Validation:

• Conduct testing and validation activities, such as unit testing, integration testing, and code reviews, as early as possible in the development process.
• Identify and address defects, security vulnerabilities, and performance issues before they propagate into later stages of development.

Continuous Integration:

• Continuous integration is the CI of CI/CD and should be implemented to automate code integration, testing, and help developers get automatic feedback on agreed rules
• Frequent integration and automated testing help catch issues early and enable rapid and reliable software delivery.
• CI helps to promote fast feedback loops

Security as Code:

• Ensure that software meets regulatory requirements and security standards from the outset.
• Incorporate security and compliance practices and testing into the development workflow.
• Define security policies as code so that security experts can declare the best practices in a way that can be checked by software
• Use tools like static code analysis, dynamic scanning and security-focused code reviews to identify and address security vulnerabilities early.

Code Quality and Code Reviews:

• Establish coding standards and guidelines for developers to follow.
• Conduct code reviews to ensure code quality, consistency, and adherence to best practices.

Collaboration Across Teams:

• Foster collaboration between development, operations, QA, and security teams so that informed decisions can be made as early in the process as possible
• Encourage open communication and knowledge sharing to address issues collectively.

Monitoring and Performance Testing:

• Implement monitoring and performance testing early the development lifecycle and make it available in the same way in production to maintain a consistent and well understood view
• Use the monitoring system to identify and address performance bottlenecks and scalability concerns early.
• Monitoring early helps to promote fast feedback loops

Source: Beyond Defaults

Benefits of Shifting Left

The benefits of shifting left are well documented, well understood and well proven in practice (remember, we're not saying it's a bad thing, just too much of a good thing isn't always the best thing). Below are some of the things you should expect to see by introducing key principles above.

1. Early Detection: Shifting Left emphasises the importance of early problem detection. By incorporating testing and validation from the project's inception, developers can spot issues before they become deeply ingrained in the codebase. This not only reduces rework but also promotes a culture of continuous improvement.
2. Time Saving: Automation plays a pivotal role in Shifting Left. Automated testing, continuous integration pipelines enable rapid feedback loops and quick identification of issues. This leads to faster delivery cycles and the ability to iterate swiftly.
3. Security: Shifting Left places a significant focus on security. By integrating security testing and best practices early in development, vulnerabilities can be detected and mitigated before they become exploitable weaknesses, reducing potential risks and ensuring a more secure end product.
4. Cost Reduction: Early issue detection and resolution minimise the costs associated with fixing problems in later stages. The expense of addressing a bug during coding is notably lower than addressing it during production.
5. Improved Quality: By emphasising early validation and continuous testing, Shifting Left contributes to better overall software quality. This leads to a more positive user experience and enhanced customer satisfaction.
6. Faster Time-to-Market: Early identification of issues, coupled with automation, accelerates the development process. This translates to quicker releases and the ability to respond promptly to market demands.

By embracing these principles and practices, development teams can shift left in their software development process, delivering higher-quality software more efficiently and with fewer defects. This approach ultimately leads to better customer satisfaction and reduced development costs.

Source: Katalon

Challenges and Considerations

Getting your shift left correct will require paying careful attention to a number of concerns as your development teams adopt more. Below are some things to look out for.

1. Cognitive overload: As mentioned, the primary concern being noticed is the amount of pressure to become experts in lots of different disciplines and burning out as a result
2. Software quality: The main aim of shifting left is to increase software quality. However, if developers are having to spend more time working on technologies and becoming skilled in disciplines they're not familiar with, being able to maintain any sort of quality or consistency in the code their writing is likely to suffer.
3. Resistance to Change: Teams and individuals may resist adopting new practices, especially if they have been following traditional development methodologies for a long time. Overcoming resistance to change requires effective communication and buy-in from all stakeholders.
4. Skills and Training: Developers and team members may need to acquire new skills and knowledge related to automation, security, and testing. Providing adequate training and resources for upskilling is essential.
5. Complexity: Embedding security practices into the development process can be complex. Developers may need to learn about security vulnerabilities and how to mitigate them. Security tools and processes must be integrated without slowing down development.
6. Culture and Collaboration: Shifting left requires a culture of collaboration between development, operations, security, and other teams. Overcoming silos and promoting cross-functional collaboration can be a significant cultural shift.
7. Legacy Systems: Organisations with legacy systems may face challenges in integrating shift-left practices into older technologies and codebases. This may require additional effort and adaptation.
8. Maintaining Speed and Efficiency: While shifting left can improve software quality, there is a risk of slowing down development if not done correctly or by applying too much burden on individuals or individual teams.
9. Scalability: Shift-left practices need to scale with the size and complexity of projects. Ensuring that practices remain effective as projects grow can be challenging.
10. Measuring Success: Defining and measuring success metrics for shift-left initiatives can be challenging. Determining whether software quality has improved and development processes have become more efficient requires careful monitoring and analysis.
11. Monitoring and Auditing: most of the time when a whole load of new technologies are thrown at a team who have no experience in using them and no desire to deeply understand them, they will follow quick-start guides, so that an, "It just works" mentality creeps into the team. Areas which are most commonly neglected (because they're not covered in quick-starts) is how those new technologies are monitored for availability and performance or how auditing of events are tracked for security and compliance.

Addressing these challenges often requires a holistic approach, with someone specifically focusing on the organisational culture, skill development, tool adoption, and ongoing evaluation and adjustment of shift-left practices.  

Source: Google Cloud

Shifting down

Shifting Down departs from conventional shift-left practices of perpetuating the trend of burdening individuals with an extensive knowledge of new and complex concepts and technologies to do their job. Instead, the approach involves domain experts focusing to capitalise on available technology and providing the facility to self-serve all the things we've previously been trying to shift left and offloading more tasks onto the platforms they're already using; streamlining technological frameworks and introducing platform abstractions to allow developers to adopt the concept of shifting left without the cognitive overload. All while still maintaining our goal of polishing the software before it reaches the end user and ensuring a seamless experience.

Key principles

All the principles of shifting left are still applicable when shifting down. Below are some of the main principles of shifting down in contrast to shifting left:

Platform Engineering Teams:

• Create dedicated platform engineering teams responsible for managing the underlying infrastructure, tools, and services.
• If you're unable to create your own teams work with a provider who specialises in platform engineering and site reliability.
• These teams focus on all the points below to create a seamless developer experience and maintain platform reliability.

Maximise Platform Abstractions:

• Provide developers with platform abstractions that hide unnecessary complexity, allowing them to focus on coding and application logic rather than infrastructure management.
• Offer managed services and environments that abstract away lower-level technical details.

Reduce Cognitive Load:

• Avoid overwhelming developers with a wide range of technologies and tools. Encourage simplicity and reduce the cognitive load by minimising the number of technologies they need to master.

Opinionated Frameworks and Tools:

• Provide opinionated frameworks and tools that guide developers in making choices that align with best practices and the organisation's standards.
• These frameworks should include default configurations and settings to reduce decision-making overhead.
• Make rich monitoring and logging services available to all development teams and make them easy and quick to enable
• Provide a suite of templates and dashboards for monitoring and logging systems, so that developers are "ready to go" as soon as they're enabled.

Automatically apply Cloud-Native Practices:

• Embrace cloud-native practices that leverage the capabilities of cloud providers to simplify infrastructure management, scalability, and reliability.
• Make these practices simple to implement with configuration and profiles (no-code) approach.
• Prefer serverless/containerisation technologies to abstract infrastructure complexities and keep your software Cloud agnostic and multi-cloud ready.

Maximise Security and compliance abstractions:

• Provide developers with security abstractions that hide unnecessary complexity and maintain up-to-date best practices.
• Implement security practices and policies by design, ensuring that security considerations are integrated into the development process.
• Offer security tools and services that developers can easily integrate into their workflows.

Advantages of Shifting Down

All the benefits of shifting left are still applicable when shifting down. Below are the benefits in contrast to an shifting left only approach.

1. Reduced Cognitive Load: Developers can focus more on coding and innovation when the complexity of infrastructure management and other non-essential tasks is reduced through platform abstractions and automation. 
2. Job Satisfaction: Most of the time, software developers became developers to write code, just as with platform teams and security experts. If they're doing the thing they always wanted to be doing, they will enjoy a better work experience with fewer distractions and a more straightforward development process, leading to higher job satisfaction and creativity.
3. Flexibility and Choice: While simplifying workflows, organisations can still allow developers some flexibility and choice in selecting tools and technologies, striking a balance between opinionated guidance and flexibility.
4. Increased Productivity: By streamlining workflows into subject specific tasks, engineers won't have to context switch so much. Couple this with all teams automating repetitive tasks in the same CI/CD pipelines, the whole software development lifecycle process will move more quickly and efficiently.
5. Consistency: Opinionated tools and frameworks promote consistency in development practices, making it easier to maintain codebases and troubleshoot issues. Having experts in given fields dedicate to their areas of expertise, they can define clear and consistent processes to get the results needed.
6. Improved Security: When domain experts are implementing domain specific technologies, they have a much better understanding of risks and vulnerabilities and how to remedy them.
7. Improved Use Experience: When domain experts are implementing domain specific technologies, they have a much better understanding of how to implement them correctly, along with any monitoring and event auditing to ensure the best availability and performance from those systems.
8. Resource Optimisation: Leveraging existing technologies and platforms can optimise resource usage. When engineers have a deep understanding of these tools and systems, they usually know the best way to get the most out of them.
9. Accelerated Innovation: By removing unnecessary burdens and providing developers with tools to work more efficiently, "shift down" practices free up time and mental resources for innovation and experimentation.
10. Technological Advancements: Domain expert refinement under the "Shifting Down" strategy provides an opportunity to leverage the most cutting-edge technological advancements and capabilities available at the time of release. By deferring these technological and policy decisions to domain experts, developers can incorporate the latest tools, libraries, and frameworks to ensure that the software is fully optimised for contemporary technologies without having to spend time researching them.
11. Adaptability to Trends: Since domain experts are able to better incorporate the most up-to-date trends and technological advancements, the whole team will become more adaptable to a rapidly evolving technological landscape and align well with current market demands, security vulnerabilities and enhance its competitiveness and value to users.
12. Focused Refinement: Developers can concentrate on refining specific features, functionalities, and user interactions that have proven to be pivotal during earlier testing phases. This focused approach to refinement enhances the software's effectiveness, ensuring that it meets the precise needs and expectations of its intended audience.
13. Project Time Management: Shifting down helps teams focus on the product delivery better because many areas are now abstracted and provided-for. Therefore, project managers can have greater confidence in delivery times because the schedule is no longer disrupted by developers having to focus on non-functional requirements. This time management advantage allows for a smoother transition from development to deployment, enabling teams to meet their planned milestones and launch dates more consistently.

Challenges and Deliberations

While the "Shifting Down" approach holds immense promise, just like purely shifting left, it is not without its challenges. Things like: resistance to change; culture and collaboration; working with legacy systems; and measuring the success still exist. However, their impact is somewhat reduced by the fact that the burden of them is diluted to individuals with expertise in their area, so they have usually have less to do to overcome them. Below are some additional points.

1. Balancing Abstractions: Striking the right balance between providing platform abstractions and allowing developers some degree of flexibility and choice can be challenging. Overly opinionated frameworks can lead to resistance or limitations.
2. Tool Selection and Integration: Identifying the right set of tools and technologies for creating abstractions and automating tasks can be challenging. Integrating these tools seamlessly into the existing development workflow is also a complex task and may still require some additional effort at the offset.
3. Customisation Needs: Some projects may require customisation that goes beyond what managed services and abstractions offer. Balancing the need for customisation with the benefits of simplification can be challenging.
4. Upfront investment: Unless you already have the domain experts, they may need to be sourced and those individuals may require some time to get everything up and running smoothly before seeing the investment paying off.

Conclusion

In reality, a balanced approach could be the most effective strategy. Some projects might benefit from "Shifting Left" to ensure early quality, while others might find value in "Shifting Down" to accommodate real-world user experiences and optimise for performance. Hybrid methodologies that integrate elements of both approaches can also be successful, depending on the project's specific needs. One way to look at solving this problem (particularly if you're looking to grow your business any time soon) would be to look at managed experiences - companies who can embed platform engineering and SRE teams into your workflow and adopt opinionated tools and frameworks that abstract away the complexities of technology management, enabling developers to focus on coding and creative tasks. Ultimately, the decision should be based on a thorough assessment of your team's capabilities, project goals, and the specific challenges you anticipate. Flexibility and adaptability in choosing and integrating these strategies can lead to a development process that's well-suited to your context.

Mesoform has been providing platform engineering, DevOps and site reliability solutions for nearly two decades and has worked with startups and some of the largest companies on the planet in doing so. Shifting down (or service oriented organisation as we've previously called it) is exactly we've been doing for at-least the last decade. This experience and expertise places Mesoform in a unique position to enable businesses to adopt a shift down approach quickly and easily and with significantly reduced challenges and deliberations. If you would like to discuss any of these topics in more detail, please feel free to get in touch