Life's random bits By b1thunt3r (aka Ishan Jain)…
Mega repo in Azure DevOps

Mega repo in Azure DevOps

Ishan jain
For sometime, I have been trying to figure out how I can run a single repo (aka mega repo) with multiple projects. The concept in theory seemed quite simple, but when I tried to implement it, it took me several attempts to even understand the whole concept.

Disclaimer: Currently I am employed by Microsoft, but my views and thoughts are still my own. The reason I joined Microsoft was, the work Microsoft have been doing for last couple of years in Open Source Space. Today I am a advocate for Open Source representing Microsoft.

Update 1: Updated to use YAML instead of Classic Pipelines

Update 2: Have had this post as a draft for a long time. Decided to publish in the current state as this does discuses some interesting topics. This does include some 'ancient' technics, but in the whole it is still relevant. I was planning to publish this post in March 2019, only 5¾ years late!

It was not before I tried to implement mega repo the first time, I realized how complicated this really is. I think it took me almost 2 years to come up with a solution I was happy with.

Then one day I saw a feature in Azure DevOps (formally known as Visual Studio Team Services), which I had never thought about before. It took me couple of years before I realized what I could do with this feature.

Build Pipeline

I know this option has been there since the start, but I could not really see the usage of it until I had my little eureka! moment. The feature I am talking about is Path Filter under PipelinesTriggersContinues Integration.

With the help of Path Filter you can create multiple pipelines that will trigger on changes in different part of the repository. Think different projects.

Yaml based pipelines have trigger with path filters.

trigger:
   branches:
      include:
      - main
   paths:
      include:
      - ProjectA

Release Pipeline

You might be thinking. While that is all fine with build pipeline, what about release pipeline?
You can setup a release pipeline to be triggered only on a artifact drop from build. Each build pipeline should have its own release pipeline.

Yaml based pipeline now have capabilities to have multi-stage pipelines.

Setup a Mono Repo

In the following scenario I am going to use .NET Core, you can use any other platform/language you want to use. Most of the steps will be similar.

In the example below, I am going to just create a release package, to demonstrate the concept.

Prerequisites

The Steps

  1. Create a new project in your Azure DevOps Organization

  2. Clone the new repository to the local machine

    git clone <repo-url>
    
  3. Create two new folders

    mkdir projectA
    mkdir projectB
    
  4. Create new projects in the folders

    cd projectA
    dotnet new mvc
    cd ../projectB
    dotnet new console
    cd ..
    
  5. Stage, commit and push the changes

    git add .
    git commit -m 'Initial Commit'
    git push master origin
    
  6. Create new build pipelines

    1. ProjectA

      1. Create Yaml file

        # ProjectA\azure-pipelines.yml
        trigger:
           branches:
              include:
              - main
           paths:
              include:
              - ProjectA
        
      2. Create Pipeline

        az pipelines create --name 'ProjectA' --description 'Pipeline for ProjectA' --repository SampleRepoName --branch master --repository-type tfsgit  --yml-path ProjectA\azure-pipelines.yml
        
    2. ProjectB

      1. Create Yaml file

        # ProjectB\azure-pipelines.yml
        trigger:
           branches:
              include:
              - main
           paths:
              include:
              - ProjectB
        
      2. Create Pipeline

        az pipelines create --name 'ProjectB' --description 'Pipeline for ProjectB' --repository SampleRepoName --branch master --repository-type tfsgit  --yml-path ProjectB\azure-pipelines.yml
        
  7. Test the pipelines

    1. ProjectA
      1. Make changes
      2. Stage, Commit and Push
      3. Verify
    2. ProjectB
      1. Make changes
      2. Stage, Commit and Push
      3. Verify

Conclusion

Now you should have two different projects within a single git repository. If the filters are working as they should, you can control which of the projects gets released when.

Resources