Transcript for Episode 13 – It's great if you're an advocate
Our podcasts are best experienced by being listened to. These transcripts are generated using speech recognition and human editing. There may be errors.
Today’s speakers are, in order of appearance:
- Ryan – Trainer at Instil. Host and Editor in Chief for the WeekNotes podcast.
- Eoin – Principle Software Engineer at Instil. Fastest man at Instil on two feet (accurate at time of recording)
- Emma – Software Engineer at Instil. International beach volleyball player and neutral Swiss resident
Ryan Hi folks, Ryan here, you’re listening to weeknotes by Instil. You may have noticed a gap in transmission last week. It turns out that I find recording and editing a podcast every week, while continuing to do a full-time job pretty difficult.
This week, I’ve gone back to your recording from one of January’s episodes.
You’re about to hear the second half of the conversation I had with Emma and Eoin. It’s mostly a single topic conversation sparked by a link we shared on our internal Slack about queuing theory. I’ll add the link to the show notes for you to read yourself.
Let’s get straight to it.
The statistics one.
I did this afternoon go back to that queuing problem. . And I think I can recap that quickly. So if you’ve got, one teller in a bank who can deal with one person every 10 minutes and you get people who arrive at a little bit longer than once every 10 minutes.
So like say 10 minutes and 30 seconds, counter intuitively five hours of waiting time on average for anybody that’s in that queue. I, downloaded R studio and a library and ran a simulation on it to verify that actually that assertion was correct.
And it turns out that it is and I don’t really understand it.
Eoin Yeah, it’s, kind of bizarre. It’s very counter-intuitive . So the customers are arriving at a rate slightly slower than the bank can deal with. So in theory, you would think what the bank can keep on top of the rate that these customers are coming in, obviously the first customer to walk in the door in the morning is dealt with straight away.
But if it runs until it reaches a state where customers are being dealt with, as they’re coming in, then the queuing times, like five hours or something,
Ryan Yeah, Eoin yeah, , it’s a crazy one. Ryan it is. It’s totally mental. Emma, you were a maths teacher weren’t you, before being a software developer, Emma I mean , I didn’t do a lot of statistics in uni cause I kind of don’t like them. And so it didn’t teach them much either, but , because it’s like a real world situation to try to apply a linear, or like a mathematical. Theory to it. It’s like, no, because , it’s too much random in it I don’t like it. Ryan I suspect it’s the randomness actually, , that makes it so weird. So , if everybody turns up. On the button, 10 minutes and 30 seconds apart, then the teller can deal with them perfectly. And everybody just has the zero queuing time. But , if you’re a long way off the average. So, if you arrive two minutes after the previous person, then you gotta wait for 20 minutes. And in that time and other three people might turn up. I think that’s , what makes the queuing system weird that there’s zero Slack in it, Eoin Yeah, so the randomness is what can actually cause, it to balloon out of it. But then I guess when you bring that back to real life, like if we can add randomness to an algorithm, But then we can say that there’s no such thing as true randomness. . So if we’ve got an actual bank with customers coming in , there’ll be lulls and be based on, , morning times lunch times, break times. People leaving the office or whatever.
So I guess the lessons could be directed towards , a server dealing with requests. So what’s the capacity of your server. I guess in that case, it is a higher workload, so that things do they will reach that steady state.
They’ve got more chance to reach a steady state and it’s more realistic that that will happen. Ryan Yeah. So I guess the question is, , as a software developer, do you need to know all of these statistical things to figure, out. How many servers you need for your application or do you just say to Amazon, you solve that problem for me? Eoin good question. Emma Yeah, you can always use the auto scaling, which is quite nice , that’s what we use in our product. If it’s getting heavy and it’s working hard, just, , pop up another one, off you go and then once it’s finished, wait till it’s done and just close it down again, you know? So it’s obviously very practical like that, but just talking about queues , we have really have a queue that takes in dealing with our files let’s say. And so if you get a really big file that we only have one queue, and so that’s going to take a long time.
But then somebody comes in and wants something quick. But you have to wait for that big beast to finish before you can get quick thing. So then we’ve talked about actually making another queue so that like the bank having a second teller would really lower when you think it would half it, but obviously in the theory is actually that it’s 93% better, which is mind blowing.
But what we thought is we could be even smarter because obviously with tech, we’re going to be smarter than humans. , And actually, if we knew it’s a big file, send it to this queue don’t care about it. And if we know it’s small, send it to this queue, get it done quick. And so that was kind of our thinking that we could do that as well, .
I think it can be a theory , we can use and implement in our own products. Eoin I guess it could be one of those. theories that it could be helpful to know about it. You don’t need to know the maths. You don’t need to run off to python and prove it. But if the thing comes up where, you know, we’ve got, this server that can deal with processing files at this rate on, we know that on average, the files come in at this rate slightly slower.
So you might put your finger in the air and go, yeah, this looks good. There should be fine. And so if you have a wee bit of background in this type of thing, , you wouldn’t do that. And you wouldn’t be surprised when it went, you know, it’s taking hours and hours to process your files, you know, so maybe without knowing , all the theory behind it, , that’s for the Amazon engineers that’s for for the library implementers . Ryan yeah. True. If you were to apply a serverless mindset to that queue, would you actually just spin up a queue on demand? We don’t have one queue. We’ve got as many queues as we need. And we’re just going to throw things in and let, the Lambda functions just scale to accommodate the number of items in our queue.
Is that a sensible approach? Eoin So, , I was actually speaking to , someone about this . So there’s the idea of infrastructure as a service, software as a service. And I was saying that it’s all a continuum. It’s not like you go full software as a service or whatever it is.
So. you could just, you know, rent out a server somewhere and just use it, or you can go full serverless, but , you always need to have some understanding of what’s happening. So if you, bring up a Lambda, it’s warm, it’ll respond to queries more quickly after that, when it is warm and you know that, as like any type of programming it always helps know a wee bit about the layer above and the layer below. Because it’s, it’s maybe never the case that we just say, we’ll have a lambda per request , that doesn’t actually happen in reality. . Depends on the domain too, but it could be more cost-effective to have a server running. and when, a queue that’s a certain point, spin up another server. And when the queue goes below that point to take away that server, that might work for one application for another application, it could be a case of . , well the extreme example is just let it fire up a serverless per request and deal with them one at a time, but you, you probably would never really want to do that.
So I think it’s an understanding of the domain. And when we were talking here, it’s hypothetical and then an understanding of the underlying technologies and an understanding of the difference between, , whatever it is, ECS or some sort of auto scaling technology. And , AWS Lambda is another example, or running up your own containers and sort of rolling those up manually or whatever it is. Ryan Yeah. I think a lot of developers like to focus on the code, I think a lot of people like to come into work and sit down and pop up the IDE and start writing their Java or their Kotlin and not think about that sort of stuff, but actually , the way you’re describing it, you need to be thinking about that stuff. All the time. You can’t write code without thinking about how you’re going to deploy it. Eoin I guess someone needs to, , we are software engineers we like correct code. I’m going to send a code. We can understand it and we can think comfortable. So it can be quite comfortable for us , , to lean towards creating that monolith, because now we’ve got this service and we can see all the files and the folder structure to the left hand side or whatever it is. And we know where everything is. Whereas when you go into a more serverless world, okay, now you’re using a queue , you’ve got some sort of retry mechanism you’ve got somewhere for, , dead letters.
You’ve got a serverless. Technology to pick up items off that queue. You’ve got event streams. It’s, it’s much harder to grasp everything, but I think because it’s comfortable to work in the monolith.
You can lean towards that, but that doesn’t necessarily mean that there’ll be a better solution. The serverless solution might be more difficult to grasp, but as long as you understand all the bits in isolation and then, you understand how they connect together you can get a more resilient solution and a more scalable solution. So, , forget about that discomfort of not being able to grasp everything and the one place and just kind of to embrace that, I guess, because with either solution, you don’t make the complexity go away.
You may feel more comfortable with the monolith, but you haven’t made the complexity go away. You just have it in a place where you feel like you can grasp it Emma testing it’s just so much harder as well though. If you’ve got a lot of serverless stuff, , like you said, it’s all a bit here and there. Whereas when you just have it all there and it’s just sitting, like you said, to your left, it just makes sense. . Eoin Yeah, , I can really, really sympathize with that. And that was probably one of my biggest criticisms of the whole serverless advocacy, especially earlier on is that yeah, you can do, these things. You can create a service, you can deploy it, it can be resilient and scalable, but in reality, we have to write tests for all of this stuff and testing’s an afterthought. And IDE integration is an afterthought refactoring as maybe an afterthought. So it’s great. If you’re an advocate and you’re going to spend two days, you know, preaching about these things and firing up your hello worlds, but when you get down to the day to day, And then week to week, and now you’ve been using it for, you know, four months, six months, these are the things like the everyday things like writing tests, doing a , refactor, those are the things that are really difficult.
And yeah, they, I think they were definitely lacking a couple of years ago there. I think they’re bad, or, but they’re still, there’s probably still a way to go. We’re quite into serverless in the current project and were writing quite a bit, of boilerplate in order to get tests, to work, to get local development, to work, you know, , , sometimes I say, I want to fire up my front end and the back end.
I want to have it all local. So that I can hit break points in the backend. And then, you know, the response is always, well, you know how you want your APA to pave so you can write a test for any type of scenario. You know, if you’re going to click a drop down on your front end, that’s going to make a call to the back end.
You can write an integration test that has that payload, that hits your backend . And then you can run that locally and debug there, . And I think that makes sense. And initially I find it difficult to argue with that, , argument to sort of argue back, but I I’ve worked in this day to day. I really find that like a good example is you’re doing a code review of someone’s code and you want to see what they’re up to easiest way to do that sometimes is to check out their code.
But if you’re in a serverless thing, you need to check out their code and deploy it. and that’s like, what? There’s that 10, 15 minutes. , or maybe you want to jump back through your history to see when a regression came in. And again, you can’t do that easily. If you need to jump to commit deploy it, okay. Try the next commit deploy it.
That’s not a realistic thing. So this thing have been running locally. It’s kind of easy to hand wave it away. But I think when you’re doing this day to day, it’s a very important thing. Very important part of the development.
Ryan Yeah, , I know Amazon have a cloud-based IDE, which I have not looked at and I know like github and on that are now rolling art, visual studio code in the cloud and all that sort of stuff , do you think that potentially has the solution that we actually, get to the point where. We’re not actually developing locally at all.
So that running containers on our developer machines doesn’t happen anymore. And we do everything on the cloud. So , we’re back to thin clients, and, everything’s just a terminal that connects into the mainframe that Amazon are hosting for us. Eoin I think it could be one of those things that the blurb sounds good. In practice. It may be good someday, but it’s, it’s not there today. Again, I’m thinking a lot of these things can sound good, but when you get down to the day-to-day tasks that all developers say, I want to check out someone’s branch and run it up and see what’s happening.
I want to hit break points on it. I want to go back through find a regression I want to flick through the git commits. You know, can you do that easily? Can you do that as easily as if I’m just running up spring boot locally? I think, yeah, those things can sound good in the blurb. And then in reality, they, , may do the 80% for 20% of the effort, but then they need it’s that last twenty percent that makes day to day development a better experience. Ryan Yeah, so that the demos worked really well. The hello worlds are amazing Eoin the demos work really well. Yeah. I’ve described this in the past as like, if you’ve ever been in a car garage and you’ve opened the door of the car and then you close it, it just sounds great. It’s just one of the best noises the noise of a car door closing, in the the inside of car, garage indoors.
And so the really put a lot of effort into that. It doesn’t say anything about how the car is going to perform in, any useful function for the rest of its life, but it’s, it’s actually a pretty important thing and it can sell the car. So, yeah, I think that’s the analogy I use for a lot of. And I’m going to, I’m going to single out AWS.
I think there could be the most guilty here. And if you read the front page and then any of the services, it’s like, it’s just going to cure all your problems.
Ryan but it doesn’t
Eoin Gives you
Emma We couldn’t really follow too much of a serverless, , structure because the product we’re design has to also work on prem. And so. We had to sort of steer away from, going too fancy on the serverless side. , so you’re kind of me glad, glad of that. In the end at first I was like, Oh, this is so annoying thinking about .
Two things always, you have S3 or on prem, basically where the files going to be. And so it’s kind of annoying on the surface, but now I thinking about it, maybe it’s a better thing. Eoin Yeah, but I would say like, overall, we. There were problems, but hopefully they’ve got solutions that are, that aren’t, you know, they’re pretty good. And we’re getting them, took this where , it is pretty good. , and then we were getting the benefits of serverless as well. So it’s always, it’s that thing of like, if you’re an early adopter, you’re also kind of a Guinea pig you do stand to gain the benefits, but you might have to go through a bit of pain too. I don’t want to sound too down on it because then the client doesn’t, doesn’t go where we were told this is all great. Ryan Yeah. I mean, it, it is awesome. , The idea of like pay per request and the ability to just go, I need something I can reach out and grab it. that is amazing. But , it doesn’t make the amount of work that we have to do any less. . The work is different, but it’s not, it’s not less easy.
It still requires a lot of focus and time from a developer. , so you know, the, pay per request might result in lower infrastructure costs, but your . Development costs, I presume are still as significant. Eoin uh, yeah. And and that’s, that’s the debate, right? So, you can easily look at and say, well, it takes you, a bit longer to set that up. And you know, you’re trying this technology, that’s not as mature, so that’s cost more development time. But then the flip side is . We’re not worrying about infrastructure.
We’re not paying for operations and we don’t have to worry about deployments. , there’s so many things that. The serverless infrastructure that can be taken out of your hands. So a lot of it’s like, yeah, you don’t see the problems that you don’t have, so yeah, it’s, it’s, it’s a difficult one to figure out, , but I think overall, yeah, we’re, that’s the way it’s going.
Right. That’s where we’re going towards serverless.
Ryan: Thanks for listening. Everyone. Join me next week for a discussion about side projects with Instil’s two Matthews.
Thanks to Amy for producing these.