Mega repo in Azure DevOps
Disclaimer: Currently I am employed by Microsoft, but my views and thoughts are still my own. The reason I joined Microsoft was, the work Microsoft have been doing for last couple of years in Open Source Space. Today I am a advocate for Open Source representing Microsoft.
Update 1: Updated to use YAML instead of Classic Pipelines
Update 2: Have had this post as a draft for a long time. Decided to publish in the current state as this does discuses some interesting topics. This does include some 'ancient' technics, but in the whole it is still relevant. I was planning to publish this post in March 2019, only 5¾ years late!
It was not before I tried to implement mega repo the first time, I realized how complicated this really is. I think it took me almost 2 years to come up with a solution I was happy with.
Then one day I saw a feature in Azure DevOps (formally known as Visual Studio Team Services), which I had never thought about before. It took me couple of years before I realized what I could do with this feature.
Build Pipeline
I know this option has been there since the start, but I could not really see the usage of it until I had my little eureka! moment. The feature I am talking about is Path Filter
under Pipelines
➔ Triggers
➔ Continues Integration
.
With the help of Path Filter
you can create multiple pipelines that will trigger on changes in different part of the repository. Think different projects.
Yaml based pipelines have trigger
with path
filters.
trigger:
branches:
include:
- main
paths:
include:
- ProjectA
Release Pipeline
You might be thinking. While that is all fine with build pipeline, what about release pipeline?
You can setup a release pipeline to be triggered only on a artifact drop from build. Each build pipeline should have its own release pipeline.
Yaml based pipeline now have capabilities to have multi-stage pipelines.
Setup a Mono Repo
In the following scenario I am going to use .NET Core, you can use any other platform/language you want to use. Most of the steps will be similar.
In the example below, I am going to just create a release package, to demonstrate the concept.
Prerequisites
- Internet connection 😝
- Git
- Azure DevOps Account
The Steps
Create a new project in your Azure DevOps Organization
Clone the new repository to the local machine
git clone <repo-url>
Create two new folders
mkdir projectA mkdir projectB
Create new projects in the folders
cd projectA dotnet new mvc cd ../projectB dotnet new console cd ..
Stage, commit and push the changes
git add . git commit -m 'Initial Commit' git push master origin
Create new build pipelines
ProjectA
Create Yaml file
# ProjectA\azure-pipelines.yml trigger: branches: include: - main paths: include: - ProjectA
Create Pipeline
az pipelines create --name 'ProjectA' --description 'Pipeline for ProjectA' --repository SampleRepoName --branch master --repository-type tfsgit --yml-path ProjectA\azure-pipelines.yml
ProjectB
Create Yaml file
# ProjectB\azure-pipelines.yml trigger: branches: include: - main paths: include: - ProjectB
Create Pipeline
az pipelines create --name 'ProjectB' --description 'Pipeline for ProjectB' --repository SampleRepoName --branch master --repository-type tfsgit --yml-path ProjectB\azure-pipelines.yml
Test the pipelines
ProjectA
- Make changes
- Stage, Commit and Push
- Verify
ProjectB
- Make changes
- Stage, Commit and Push
- Verify
Conclusion
Now you should have two different projects within a single git repository. If the filters are working as they should, you can control which of the projects gets released when.