How Pandadoc CEO Mikita Mikado grew Pandadoc to $100M revenue and 30K customers in 2024.

hi everyone my name is Sergey bearsuke and I'm CTO and co-founder of panda doc and today we're going to be talking about how we are doing this continuous delivery of the product my talk is going to be slightly different from the business talks because I'm going to be talking more about the production side that's what I do and I manage product and Engineering so that's why sometimes I have arguments with myself yeah but so 800 product updates you might call BS but I checked today like how many did we do today 24 so far they is still going and so that's all the updates we're delivering to production which our customers can can already use the question is why would you care about this and so let's take a look at the product life cycle and my assumption here and hunch that many of you Founders and even ourselves uh we are playing in this two first stages introduction and growth right what's important in those two stages is that so we need to learn a lot and iterate to in the first stage to find this product Market fit to test ideas to validate things and with the growth stage so we need to close this Gap really quickly to deliver the value to the customers and actually get this uh the leader position on the market right what do we do to do this is we're going through this continuous learning cycle right so we have a hypothesis so we do something we launch this we test and validate we measure then we iterate and change the speed of this continuous learning cycle depends on the speed of the slowest Link in the cycle and the problem sometimes problems there in the different stages but the most of the problems you see they lie between the implementation in review and the reason why that so your idea in initiative is not just only one in your company and when you're ready to push this out to go and test and validate so what you face is tons of things is happening in the product and you need to integrate this in the code and you need to merge you need to talk to all the developers product managers make sure that it's all integrated but then you also need to see together package the whole thing deliver this to production and so you need to go through the testing you need to go through the deployment and all the stuff right what it does essentially it's actually slows down this learning cycle because you don't know if it's going to stick or not and you need to iterate and you need to do this quickly so that's why people came up with this concept of continuous integration and continuous delivery and it's also known as CI CD you probably heard about this right and in many many cases people actually talk about more of engineering side of the story there are other sides of the story and we'll Sprint through engineer inside really really quickly just to explain how it works and we'll focus more on the product and other sites so with the engineering the way how it works is the whole concept is so this is courtesy of gitlab and how they explain continuous integration continuous delivery that so when we start working on something so we just create a branch and we're continuously delivering those little updates onto the master branch which is the latest version of the application but when you do this it goes through the pipeline and it automatically tests and do all the steps needed in order to integrate it so you make sure that it's not going to break anything it's not gonna cause any issues etc etc and what I mean by this is the steps in pipeline it's a testing automated testing end-to-end test security packaging linters called style and all that stuff and the next stage is the continuous deployment so what is that is like you need to package also your app in order to deliver to production containers libraries uh updates whatever whatever you need to do there's also automatically so when it helps you to it helps you to eliminate this bottleneck on the engineering side to launch this really quickly and accelerate the cycle this is the real live example of panda doc one of the pipelines one of the services so what would you do we automate all the steps security end-to-end tasks packaging linters called Styles and everything so when you do this and when you merge like it goes through this Pipeline and you kind of sure that it's not going to break anything so a couple of things what we'll learn so far and if I would give just one advice to you guys uh if you're interested start doing this as early as possible because it is much easier to do this in the very beginning than later apply to to everything because if you have like 200 services and you need to apply this to all the micro servers and all this other just such a big project um to to take that sometimes people are Hazard element to start and a couple other things but the module architecture Loosely coupled system will definitely help because it will again decouple the whole thing and you can independently launch and update things in the app now let's move into the product side of the story and the the product side of the story is sometimes is overlooked because I mean engineering will figure this out and they're going to be launching this but um it's actually even I think more important than on the engineering side very very high level those are type of updates you are producing um in your probably day-to-day jobs so to update the product so you're fixing some bugs so you're doing some technical speed improvements you're doing some ux improvements your opposal launching some features and all these things so the first two is like you probably deliver to the customers as soon as possible because you want to you know scores this bug and move on right the latter two they are more complex because sometimes you don't know what you're doing and you need to test and again that's where you apply this continuous learning cycle that's what we'll learn right so the product manager is um have a hypothesis so we're going to test this we're going to launch this we're going to measure and how it's going to affect everything what I find very often in the mindset we're constantly thinking about the end stage right controversial image I know and this is from agile world but I think it's actually a good representation the way how you might think about this is like when you think about the feature you're thinking about this end stage and you can do continuous integration continuous delivery all day long but if in the middle of this process your customers can't use it you're not learning anything if you're waiting like you're doing continuous delivery every day and update but the customers can't use it for the next three months it doesn't help you to learn right and so the mindset should be changed in order to switch to this you know learn by iterating approach and sometimes it's a little bit hard because sometimes when you do an iteration and then you need to take two steps back remove something rework something this waste is create this you know like tensions oh I don't want to do this but sometimes it's needed so and I think it's a um mindset shift should should happen like on the product management side now imagine everything is working great so your developers delivering this code continuously you are product managers are thinking about this and iterations like slicing and dicing this correctly and so you have this amazing update uh and it went live through the continuous delivery do you want to show this skate to 100 000 customers all together probably not right so you're not ready for prime time so you need to test this thing first with a very very limited subset of the customers maybe get their feedback maybe iterate when you're ready for prime time then have your prime time and in order to manage this scale people came up with the idea of the feature flag so what is feature flag feature flag is basically a toggle which tells you this if this thing update change feature ux update is visible to a particular customer and in order to manage this on scale not in the code there are different systems which allows you to do this and Define those feature flags and independently manage them from the code so when you write your feature you wrap this in a feature flag and there is a system there is a split IO real system and you can use it to manage those stacks and gradually roll out the features to your customers when you need it like to learn to get the feedback to see behavioral metrics and all the stuff so the panadoc we probably right now have roughly 300 feature Flags like in a different stages some of them they're rolled out completely some of them they're like steel maybe like one two percent but this is how it works and the product managers can actually do this uh changes independently from the system so I don't need to release anything so it's independent and if something goes wrong they also can go down to zero and nobody will see this uh our strategy about the audience and the rollout strategy is kind of like very very slow in the very beginning so we take this only internally we test this so we have teams who are using panda dog every day so they can give feedback and we're gradually rolling out this to other users so we like to work with the customers we know and we trust so it's a cap customer Advisory Board some early adopters some recruited users who might be interested in in this feature but then we are ready for prime time so we start testing on a little bit bigger audience and the tendency is and a strategy we like to use as like lower risk to high risk because you probably have also diversity with the customers in the very beginning so you might go with the maybe I don't know like a free product customers or something and then switch to...