Azure DevOps Tutorial for Beginners | CI/CD with Azure Pipelines
TechWorld with Nana
https://www.youtube.com/watch?v=4BibQ69MD8c

Transkript (automatisch erstellt)

0:00 so in this video we will learn about one of the popular devops platforms called azure devops we will see what it is and what you can do with azure devops
0:09 platform how to use it for your software development projects and generally how is this relevant for you as an engineer we will see different features and use
0:18 cases of azure devops how it all works and how you can implement the whole software development and deployment lifecycle with it we will also review
0:27 the azure devops architecture and how it works in the background and finally we will also compare it with alternative tools and talk a bit about which of
0:37 these tools you need to learn for your career with so many alternative options so let's get started so first of all what is azure devops as the name
0:47 suggests it is a devops platform it is a software as a service offering which was created to basically be a one-stop shop for implementing
0:58 all your devops processes for your project and it had many names before it became known as azure devops it was called team foundation server then
1:08 visual studio team services so it had a bunch of other names before azure devops because it extended from existing services and tools so it kind of got
1:17 names of these tools and even though it was for the same purpose of implementing the devops processes at that time devops was still practiced by a smaller group
1:28 of projects so that's why the name devops never actually came up in the name of the platform itself however since devops became more mainstream and
1:39 as it is now used in lots of projects worldwide it was actually renamed to azure devops which
1:47 is pretty smart because devops is already a popular term everybody knows what it is so just by the name you know that it's a platform for devops but it
1:56 still sounds too general because what is a devops platform devops is many things right so what is a platform for devops and to
2:05 answer that question let's look at what azure devops is exactly and how it helps in implementing devops processes well devops as i said is many things it is a
2:15 combination of concepts and tools and basically anything that makes developing and releasing applications fast in an automated way and with high quality
2:26 possible so a project needs to implement devops practices in order to achieve an efficient workflow if you want to go more in detail about devops and learn it
2:37 in more depth all its tools and concepts etc i actually have a separate video on that but shortly explained it's to make the software development lifecycle as
2:47 efficient as possible by fully or mostly automating it and azure devops is basically a technological implementation of that devops process which covers the
2:59 whole software development lifecycle and it has features for each part of this lifecycle so let's go through the steps or parts of that software development
3:09 lifecycle and see how the various azure devops features map to those parts now what does a software development lifecycle include
3:24 it's not just developing the application or writing the code it actually starts before you have written a single line of code
3:32 because before coding any feature or an improvement needs to be planned first right mostly by a product manager so the first step is to define what we are
3:43 developing and why are we developing it or in other words what's the business value behind it and there are several ways of defining the workflow of how the
3:52 project team will work on the application which roles they have within the team how they will divide and split the tasks etc
4:00 two of the most popular workflows are agile and scrum many modern projects use one of these approaches for their development so once you have created a
4:10 project in azure devops for your application the first feature you will probably use here will be azure boards and depending
4:19 on which workflow you use for your projects in your company generally like agile scrum or even some basic workflow you can choose the corresponding board
4:29 for this project because as i said azure devops or any other similar platform is basically just a tool that gives you various features to implement whatever
4:40 workflows you have in your company so on azure boards you can create tickets or tasks for features improvements or fixes for your
4:49 applications as part of that agile or scrum processes you can assign it to people to work on and you can also track progress of those features or
4:59 improvements while they are being developed now developers need to take that planned task and actually develop it right in that process they may have
5:10 questions about the task so azure boards can be used for communicating between the developers testers product owner etc within the task description plus it can
5:22 be used to have an overview and transparency over the status of the feature as it is being developed who is working on it what stage it is in what's
5:32 its progress status has it been deployed already etc now the application code that developers are creating is also part of that life
5:44 cycle actually the main part of it an azure repository feature is what you can use in order to host that code now of course you know the popular code
5:55 repositories like github gitlab bitbucket etc which are all based on git and that's basically an alternative to those platforms so editor repository
6:07 also supports git which is the most popular version control tool so developers can host their code in azure repository and push their changes to it
6:17 however the code repositories have actually evolved and became much more feature reached and just hosting code is one of the many features that they
6:27 provide so azure repository also it's not just for hosting the code as part of the devops life cycle developers collaborate and work together to develop
6:38 high quality code so in the repository as part of the git workflow you have features such as pull requests branches
6:48 various collaboration features and so on so when developer starts a task they create a temporary branch when done they create a pull request and other
6:57 developers can review and comment on the pull request they can communicate and collaborate until the pull request is good enough to be merged into the main
7:07 branch now this is what's called a git workflow so basically how your team decides to work on the code and to make sure that
7:17 the quality of code is very good and there are many git workflows or the way teams use git and all its features and different companies may use
7:27 different approaches or different workflows but the key point here is that with azure repository you have the tool that enables you to implement whatever
7:36 git workflow you choose to work with plus note that the repository and branches and the pull requests are all linked back to the feature task so you
7:48 see the activity and status of development there as well
7:55 now once the feature is developed and pull request is approved and merged into the main branch it needs to be released right that's the reason we're developing
8:05 the feature in the first place so we can release it and the end users can use it so in order to release our code changes we first need to test it and package it
8:16 into an artifact which is a deliverable that we can then deploy on the end environment and again devops is all about automating things and workflows so
8:27 that it's fast and efficient so this process of testing and building the application is done by an automated ci or continuous integration process and
8:38 for building the ci pipeline azure has what's called azure pipelines section pipelines can be written in yemo which means you can have your pipeline script
8:48 as part of your code the main building blocks of azure build pipelines are steps for example if we want to test and package the application
8:58 we may have steps to run tests package application build an image push that image to an image repository so that later it can be deployed so each
9:08 step will execute a certain command to run the test to package the application build a docker image and so on and this is an example
9:18 pipeline for building a.net application which is a common type of application built on azure devops platform since asp.net is also part of the microsoft
9:29 technology stack in the first two steps in this pipeline we test and build the application with net commands and then build the image with a docker command
9:38 and as you see we execute commands in script attribute however we have another option for executing step commands instead of
9:47 scripting it directly in yemo ourselves so in addition to writing the scripts yourself you can use what's called a task instead
9:57 so what is the task and how does it work in azure devops you have loads of tasks already available to choose from for different use cases
10:08 and they can be selected and configured using a ui you can select the tasks directly from the list of available tasks on the right
10:17 side of the pipeline's yaml editor and configure any needed parameters for the task and when you have the task configured basically you fill in and set
10:27 all the parameters you can add the task back to your yaml pipeline it will be automatically converted and edit as yaml code and of course you can adjust or add
10:38 any additional configuration to it so here for example if you look for asp.net task you will find one
10:48 for executing various.net commands like test build etc and once configured you can add the test task in place of the script you can do
10:58 the same for docker you can search for docker tasks and find the one with build and push commands and add that task to your yaml pipeline and
11:09 this obviously can be convenient in many use cases because you don't have to know the commands exactly you don't even have to know or memorize the exact syntax for
11:20 the pipeline and you also don't have to script the steps from scratch so this basically gives you a simple um an easier high level approach to adding
11:29 steps to your pipeline now this is a very simple scenario of a pipeline with one single job that has all the steps but in practice we often have more
11:40 complex scenarios where we would need multiple jobs in a single pipeline let's say we want to execute the test step on two different operating systems we want
11:49 to test the application on linux and windows before we build it or let's say we test and deploy our application on linux machine but we need
11:59 a database for our application which must run on a windows machine for this use case we would need two separate jobs now i'm mentioning jobs but we haven't
12:09 seen a job defined in our simple pipeline yet so what is a job exactly well all these steps actually belong to one job and can be defined like this so
12:19 a job is basically a group of multiple steps however when we have only one job in our pipeline we don't need to explicitly define it that's why we could
12:28 skip this in a simple pipeline but when we have multiple jobs we need to define them within jobs attribute so we have multiple jobs
12:37 each with multiple steps and now we can actually execute each job so all the steps within the job on a different environment also called
12:47 an agent so agent is a machine that will execute the tasks or the steps of the pipeline like running the test building an image etc and an agent is selected
12:58 from an agent pool like pool of windows machines or linux machines or mac os machines etc and that's why we have a pull attribute
13:09 in the job to define what kind of machine we want to dedicate for that job and as you see inside the pool we can define if the end image that specifies
13:20 what kind of operating system we want and you can also specify what version of that operating system you want to run that step or that job
13:30 another very common use case for multiple jobs would be if we have a set of steps that can run in parallel so that the build overall is faster this
13:40 could be running multiple tasks that can run at the same time testing different parts of the application so they don't have to wait for each other to complete
13:49 they can all run at once so by creating multiple jobs for all these tests you can actually run them in
13:57 parallel on different environments so overall the main task of this build pipeline is to test the code changes and if everything is fine produce an
14:07 artifact that we can deploy before moving on i want to give a shout out to polumi who made this video possible polomi is a universal
14:16 infrastructure as koto and the cool thing about polomi is that you can use your familiar programming languages tools and engineering practices to
14:26 deploy and manage your cloud infrastructure you can use polami for any cloud but for azure specifically it has the most complete infrastructure as
14:35 code support for the azure platform and is the only one with guaranteed same day support thanks to as your native provider polomi can
14:46 easily integrate into any cicd platform for azure devops they actually build a task extension that lets you easily use polumi in your ci cd pipelines it can be
14:56 used with azure pipeline's wizard ui or the yaml configuration if you want to learn more about polomi i actually have a separate video on it where you can see
15:06 fully how it works in practice with that let's move on to azure artifacts now traditionally depending on the application programming language the
15:17 artifact produced will be different could be a jar or war file for a java application a new ga file for net a zip file tar file etc
15:28 and for storing this kind of artifacts you actually have another feature in azure devops called azure artifacts and azure artifacts actually currently
15:38 supports three types of artifacts which are maven packages nuka packages and npm packages so if you're developing and building your application with any of
15:49 these tools then you can store the artifacts produced in the build pipeline in the azure artifacts however in the modern software
15:59 development we usually don't produce such artifacts anymore to deploy them instead we create docker images is artifacts so no matter what language you
16:10 use what tools you use the artifact is always the same which is a container image so if you have 10 microservices
16:18 all in different languages you can still produce the same container image type of artifacts and images actually need a dedicated
16:26 type of repository so if your build pipeline produces docker images you will connect your azure devops to some docker registry like docker hub azure container
16:37 registry etc and basically store your images in that repository
16:46 now let's say we successfully built our application into a docker image we pushed it to a docker repository and
16:55 now it's time to deploy it to the end environment this would be the next stage of the pipeline which is also called the cd or continuous deployment or
17:05 continuous delivery if you're not totally new to devops you already know that at the heart of devops there is the ci cd pipeline which is ideally the
17:16 fully automated process of taking the code changes and deploying it all the way to the production by testing and validating all parts of
17:27 those changing from is the application functioning is it secure etc and in that process we have these two main parts which is build stage and
17:36 the deploy stage again we haven't defined a stage in the pipeline yet since we only had one stage with all the build jobs when we have multiple stages
17:46 however we need to configure that as well so in order to create a complete ci cd pipeline we'll have the stages in our yaml pipeline script for building and
17:56 for deploying the application and by the way we can use a specific type of job in the deploy stage which is called deployment which is specifically meant
18:07 for deploy job and has some features for that purpose for example it doesn't check out the code like job type does etc now when we are deploying the new
18:17 application version usually we don't directly deploy to the production instead we deploy to intermediate environments we test it extensively and
18:25 gradually promote it to the production when we're almost 100 sure everything is fine common is to have development testing
18:34 and production environments so we can have all these as separate stages after the build stage deployed to development deployed to testing deployed
18:43 to production now the code for deployment to different environments will be pretty much the
18:51 same except for a couple of parameters so how can we avoid repeating the pipeline configuration code in this case or maybe we have multiple applications
19:00 that all have the same pipeline logic so we don't want to write the same pipeline configuration for each application in our company instead
19:10 ideally we want to write that logic once properly and then reuse it for all the applications that may need it in azure devops pipelines yaml syntax we can
19:20 actually put any code that is repeated and extract it in what's called a template which is a separate file and can be referenced in the pipeline
19:29 using template attribute and it can even be configured with parameters so it's like a reusable piece of configuration that you can reference in different
19:41 pipelines so you can split your entire pipeline into multiple individual files and these files can even be stored and
19:50 managed in a dedicated separate repository and as i said all the pipelines can reference them and by the way you can have a template for a job a
20:00 step or stage so for any of these levels and you can also have templates within the templates creating a hierarchy like this
20:12 now when you have multiple environments for multiple applications it may become difficult to have an overview of what version of what branch
20:21 is deployed where or when the code was last deployed to a specific environment and so on and that's where the environment feature
20:29 comes in which is part of azure pipelines we can create environments in azure devops which will map to the actual deployment environments and then
20:40 you can configure in your pipeline which of these azure devops environments you want to deploy to so you kind of have this abstraction there and once the
20:48 application gets deployed to these various environments you can actually view the deployment history per environment so this can actually be some
20:56 additional valuable ui feature that gives you a better overview of your deployments plus the deployment status can also be linked back to the original
21:05 ticket so you have that additional information for the feature or improvement to which stages or environments it has been
21:13 deployed to already which again can be pretty convenient now that deployment process or deployment part of the pipeline which as
21:24 i said is called cd or continuous deployment on azure devops can also be built as a separate pipeline called release
21:33 pipeline interesting to note that many ci cd platforms like jenkins gitlab csd etc they have one pipeline for the whole process so we have one file and one ui
21:44 unit for both in this case it will be split into separate ci and cd pipelines so the way it works is that you select a build
21:53 pipeline that produces an artifact or you choose the already built artifact location or source and you create a release pipeline for that artifact note
22:05 that release pipelines in azure devops can only be created using the ui so you have no yaml file for that however the pipeline structure itself is the same
22:15 you can create steps by choosing from available tasks and also have multiple stages like deployment to development testing and production so with release
22:24 pipelines you say when this build pipeline completes and successfully creates the docker image artifacts for example trigger this release pipeline so
22:34 this way you chain them but as i quickly mentioned as the artifact source you can actually use not only the build pipeline output but also already built artifact
22:46 from various sources now generally it's usually a better idea to always have one ci cd pipeline defined in yemel instead of splitting
22:56 that into two plus you have all the benefits of scripting your pipeline and making use of the reusable templates etc so the release pipelines is probably for
23:06 more specific use cases maybe when you want to deploy existing artifacts from the artifact repository directly but as i said usually you
23:14 should have one pipeline for the complete cicd process now an important part of an application release process is testing and you need
23:26 to extensively test your code changes before deploying it to production and of course the more complex the application the more tests you need so in azure
23:35 devops you actually have a dedicated section for tests and here you can actually create a unified central view of all the test cases or many of your
23:46 test cases that need to be checked before giving a green light to production deployment and here you can create manual test cases or plans
23:54 so when a new feature is being released a tester can go through these steps and test the application but this could also be automated tests
24:04 which will be executed as part of the ci cd process and the test reports from the pipelines can be published and viewed here
24:12 and the main advantage of this is that you have own centralized place with an overview of all the test plans whenever releasing your application whether these
24:22 are automated tests that were run in the pipeline or manual tests from developers product owners testers etc and again you can see the results of those test
24:32 executions in your feature descriptions to decide whether you can release the changes or not and you can even view and run the test cases related to a feature
24:41 directly from the convent board now the pipelines execute tasks like running tests building an image etc and
24:52 as i mentioned you can execute these tasks on different machines with different operating systems so where are those machines exactly and
25:00 how do we get access to them to run our tasks and which machines and which environment do we get to answer this question let's look at the azure
25:09 devops architecture on a high level at the core we have what's called the azure devops services a software as a service or the managed online solution
25:20 from azure and that's the main part where configurations are made pipelines and repositories are created and stored etc so these are all dedicated machines
25:31 for those things but the pipeline tasks themselves run on separate machines called agents which are connected to the azure devops
25:41 services platform now who manages these agent machines well azure offers managed agents as well so
25:49 you can let microsoft actually manage the whole setup for you including the main service which holds the configuration plus the machines that
25:58 actually execute the pipelines however in practice many companies need control over these machines plus they want to remain flexible and save costs maybe
26:08 they have these machines on premise or even on another cloud platform so they want to make use of this so you have an option to configure your own agents and
26:18 connect them to the azure devops platform or you can even have a mixture of both and this is actually pretty similar architecture to what other
26:28 similar platforms like eclipse icd etc look like so nothing really extraordinary here and since this is a managed service of course you have to
26:39 pay for these services and using for these resources but azure devops does have a free tier to get started with that basically includes a certain amount
26:48 of free resources that you can use to get started with including using managed agents for your pipeline jobs
26:58 great so till now we actually saw various features of azure devops that map to different parts of the application development life cycle
27:09 starting from planning the task all the way to developing and deploying it to the end environment now throughout these processes we
27:17 actually have tasks that we execute on other platforms for example when we build and push an image that needs to be pushed or stored in an external image
27:28 repository right which is outside of azure devops or when we deploy to a remote server it will be on some cloud platform like azure aws or even
27:38 on-premise or maybe we deploy to kubernetes cluster etc plus we may have the pipeline connected to the external code repository in github instead of
27:48 using the azure repository so for all these tasks azure devops needs to connect to those platforms right and normally you have credentials like
27:57 username and password or access token from these platforms that you need to make available in azure devops so that it can connect and authenticate itself
28:08 with those platforms now for those use cases in azure devops you have what's called service connections feature which makes managing access to external
28:18 platforms much easier first of all it's less configuration effort because you don't have to create these credentials in the respective
28:26 surveys and then replicate in azure devops instead the credentials are created automatically when azure devops connects to those services and second
28:36 advantage is that it's more secure actually because service connections use short-lived credentials which as i said gets generated on the fly when the
28:46 connection is established so you don't have to worry about rotating or invalidating credentials and so on and the service connections can be
28:54 created in the project settings section so you have a separate section for administering the project here you can manage settings for all the features
29:05 like boards repositories pipelines test plans and artifacts plus as i mentioned you can use self-hosted agents to run your pipelines and this is also
29:17 where admins can configure these agents as well now after learning all the various features of azure devops and what the
29:25 azure devops platform even is you're probably wondering if this is so great why aren't all the projects using it or is it that great and what is the
29:33 difference from other similar platforms like aws or gitlab etc and which one are you supposed to learn should you become an expert in azure devops and ignore all
29:44 other tools so let's look at comparison with similar tools and answer the question about which one to learn first let's
29:53 compare it with traditional ci cd tools like jenkins or modern ones like argo cd circle ci etc the main difference here is that these are exclusively ci cd
30:04 tools right so jenkins circle ci etc they're specifically built to create and manage cicd processes but azure devops actually strives to be the
30:16 complete devops platform not only the ci cd so any feature you need for covering the whole devops process including the ci cd is in one place which can be
30:28 extremely convenient as i said because in devops you need multiple tools for different parts of the process like jenkins for build code repository
30:37 jira board artifact repository etc which means you need to integrate these tools together so you have an effort in
30:46 putting all these tools together like connecting git repository with jenkins connecting jenkins with jira to update status of feature tasks etc so
30:57 when you use a platform that offers these services in one place obviously it's more convenient because they are already integrated
31:06 and plus you get a better traceability meaning you have links between all parts of the process feature task has links to corresponding feature branch or pull
31:15 requests to its pipelines maybe the artifact that was produced with the version and so on so you have a better overview because you have linked data
31:23 from all features a direct comparison to azure devops is however gitlab because gitlab which started off as a git repository actually made a turn and
31:36 decided to create an all-in-one devops platform as well and to be honest many of the features and use cases are pretty similar between
31:45 gitlab and azure devops or generally how the things work and if you're interested i actually have a crash course as well as full course on
31:53 gitlab ci cd for building complete devops processes with it you can also compare it with aws however
32:01 aws is way bigger and way more encompassing than just the devops processes and here i want to mention an
32:09 interesting note about comparing azure with aws and where azure devops platform actually plays a role in that as we know azure and aws are both cloud platforms
32:21 where you can create and configure your complete virtual infrastructure create virtual servers and use a bunch of other services as well but while aws has all
32:31 its services on one place with one account azure platform and azure devops platforms are more separated so you have two separate accounts for them and you
32:40 can manage them separately and even use each platform without the other however they are both obviously microsoft products part of the same ecosystem so
32:50 they have some integration so essentially they are still connected so for example the hosted runners for azure devops run on azure platform as well as
32:59 the code on azure devops repositories are also hosted on azure platform and you can also integrate the azure active directory which is one of the azure
33:09 services in your azure devops account so they're two separate platforms but integrated with each other for various use cases and that means when you want
33:19 to deploy to azure virtual machines azure app services etc or azure kubernetes service from azure devops you basically have to connect to just like
33:29 you would to any other cloud platform to deploy to it and an interesting use case in many projects many companies is that projects who use azure devops actually
33:38 deploy to multiple infrastructure environments or cloud platforms so they may deploy from azure devops pipeline to azure
33:48 virtual machines and aws virtual machines and that's probably the main example that may answer the question of which technology you should learn
33:58 or whether you should learn azure devops and the answer is usually companies that already use azure platform or already use
34:07 microsoft services and products they actually tend to use azure devops as well however in terms of cloud technologies um or cloud platforms aws
34:18 is still the winner and number one in this category and on the other end in terms of the cicd tools themselves usually the open source solutions like
34:28 gitlab scarcity or jenkins etc are preferred rather than using proprietary services like azure devops and very often you
34:37 have a case where company already has tools and processes it has been using for years like they work with jira and jenkins and aws platform they use
34:46 internal docker registry and github repository so they want and they can't just move everything to azure device so as an engineer when learning new tools
34:57 one of the things you should ask yourself is when you get a job what tools you will most likely be working with based on
35:06 which tools are mostly used by companies today and as i said tools like jenkins aws platform gitlab or github platforms etc in devops are still the most
35:17 commonly used ones and the leaders in their own categories so you should definitely look at those tools first and since many viewers
35:26 actually ask that that's the main reason why in our devops educational bootcamp we teach exactly those technologies which are the most popular and most used
35:36 in each category even if there are cooler or better alternative tools for those technologies but again some companies that already
35:45 use microsoft services may decide to go for azure devops or you may be in a project or interviewing for a job where you need this knowledge in which case of
35:55 course you should learn and get expertise in azure devops now i hope i was able to give you clarity on what azure devops is and give you all the
36:05 needed information to get started with it i will add any relevant links in the video description so you can check them out there as well as share in the
36:13 comment section whether you already have experience or have worked with azure devops or what is your experience with any comparative
36:21 tools and which one you would recommend and with that thank you for watching and see you in the next video