The Joys Of Programming

I wish my company would embrace containers, even if just for development purposes. Server instances would be much easier to get up and running. But their overly complicated design has made it nearly impossible for me to do it myself without fairly major changes in architecture.

As an aside, when I was running a Valheim server earlier this year, I used docker-compose to deploy the server.

I’ve used docker-compose quite a bit for projects with dozens of containers. I’m gradually trying to move to using k8s for that. I’m looking at Skaffold as an option to ease that transition.

If your dev stack starts to get very large instead of running it all locally in k8s there are tools like Ambassador Telepresence that let you run some local containers as if they are part of a larger remote k8s cluster.

I haven’t tried Telepresence yet but it’s on my list of things to try to improve our dev experience.

qaraq wrote:

I think just about all the other devs at work run/test apps through Tomcat that they control through intellij. I instead set up a Docker stack with our DB, Redis, nginx, and whatever tomcat instances I need for what I'm testing. It just seems a lot cleaner to me, and it's easy to get the apps talking to each other if I need to.

Perhaps, but then the IDE has to attach to the Tomcat process to debug it and any web application that Tomcat is hosting. That's still incredibly messy, especially if the Tomcat container isn't set up to allow for hot swapping class files and application contents as part of development.

As a software developer with all sorts of workflows already defined, bringing in Docker support (and teaching that to other devs) has been nothing but frustrating. Sure, I have written shims around it to help alleviate the growing pains, but as someone that already has to know how to get sh*t installed and working on my system, alongside how to run every other tool directly on the codebase, obfuscating that into a container just adds more fuel to the frustration.

"Create workflow aliases to work with the containerized apps", people say. Well, then it's either a double set of commands to remember (for containerized apps, and for non-containerized apps), or move everything into a container.

I'm not saying it's bad, I just want it to be so much better.

IMAGE(https://media1.giphy.com/media/3o84sF9H8TXFbkJC0M/giphy.gif?cid=ecf05e47qjr7gtuh5kp375j4bd03y7nzihakhtrjfv4r07en&rid=giphy.gif&ct=g)

We let pandasuit talk a little devops and look what happens...

EDIT: Spam message got deleted, now my post makes no sense.

trueheart78 wrote:

or move everything into a container.

I'm not saying it's bad, I just want it to be so much better.

That seems to be the end goal. Once you start containers can't go back

*Legion* wrote:

now my post makes no sense. :)

Don't worry, we're used to it.

IMAGE(https://i.imgur.com/rXoc4s3.gif)

Anyone here done much work with real time event based microservices?

I've used Kafka as a messaging system for cases when durability was more important than latency. If I want a message to arrive precisely once but I care far less about end to end speed: Kafka.

In most other cases over the past 10 years I've been building brokerless event systems on top of ZeroMQ. It can be very very fast but, especially with patterns like PUBSUB, you can run into port exhaustion on any decently large cluster and you end up having to split into smaller clusters. Not really worth ragging on ZeroMQ. It's been mostly good to me but I've spent way too many hours of my life coping with it's limitations.

Lately I've been looking around for alternatives. Something with very low end to end latency. I've noticed a few places recommending Redis PUBSUB and Streams. I've used Redis for some very high write rate and low latency systems in production before and I trust it a lot for the use cases I've tried. Roblox claims to be using Redis PUBSUB for very high rate notification systems.

Anyone here worked with Redis PUBSUB or Streams as scale? How about alternatives? Any experience to share?

RabbitMQ is what one of our apps uses. I know part of the team is trying to figure out replacing it in AWS but I'm not sure what they're up to

Stele wrote:

RabbitMQ is what one of our apps uses. I know part of the team is trying to figure out replacing it in AWS but I'm not sure what they're up to

Amazon’s hosted RabbitMQ is called Amazon MQ.

If you hear anything about their experience with Amazon MQ I’m curious. I haven’t tried it yet.

Sh*t that is absolutely pissing me off today:

IMAGE(https://i.imgur.com/OUKX4O7.png)

IMAGE(https://i.imgur.com/HMzp1ik.jpg)

EDIT: Oh lord. Elsewhere, in a different API call:

IMAGE(https://i.imgur.com/QC5T1ys.png)

IMAGE(https://i.imgur.com/GRBw4xZ.jpg)

I am so disproportionately angry.

Good news is that the jury would acquit your homicide.

So are we not going to comment on the mixing of lowercase_underscored and completelylowercase conventions within the participants array?

That reminds me an awful lot of my brief experience getting access to an EMR API for a research project. Different calls would return identical field names with very different types of data. Or a user ID would be userID in one call, user_id in another, and just user in a third.

I came away feeling like it perfectly encapsulated the hellhole that is American healthcare and tech.

I saved the triple Spidey image for later when something like this inevitably crops up for me and I need solace.

I named it "session-id.jpg".

merphle wrote:

So are we not going to comment on the mixing of lowercase_underscored and completelylowercase conventions within the participants array?

Also not pictured is how inconsistent the use of lowerCamelCase is.

For example, what would you figure the lowerCamelCase version of "preferredfirstname" would be?

You guessed it: preferredFirstname

But don't worry, the "n" in "name" gets capitalized in some other fields, just not that one.

It’s nice that I can inject a sidecar into a running k8s cluster to add functionality to running pods without the pods noticing. Need encrypted connections between pods with mTLS? Inject LinkerD.

It’s not so nice that I can inject a sidecar into a running k8s cluster that totally messes with the functionality of running pods and break the system in unexpected ways. Want to ruin your p95 latency by adding the occasional 50ms hiccup and listen to a bunch of consuming apps scream about timeouts and inconsistent connections? Inject LinkerD.

Edit: no this wasn’t in prod

Edit again: figured out the issue. Not LinkerD’s fault.

Today:
- this code is terrible. why would anyone do a blocking IO call in this asynchronous service?
- check git blame
- shame: oh I wrote this code years ago. why would I do this and how did no one notice this affecting production for this long?
- check why code is being used in an async service in first place
- relief: oh ok this code was written as part of the synchronous script system where it could not be async and the person who wrote this service leveraged synchronous code they shouldn’t have
- confusion: but how on earth did no one notice this affecting production for this long?!?
- fix

pandasuit wrote:

It’s nice that I can inject a sidecar into a running k8s cluster to add functionality to running pods without the pods noticing. Need encrypted connections between pods with mTLS? Inject LinkerD.

It’s not so nice that I can inject a sidecar into a running k8s cluster that totally messes with the functionality of running pods and break the system in unexpected ways. Want to ruin your p95 latency by adding the occasional 50ms hiccup and listen to a bunch of consuming apps scream about timeouts and inconsistent connections? Inject LinkerD.

Edit: no this wasn’t in prod

Edit again: figured out the issue. Not LinkerD’s fault.

Related to this: some of my colleagues over in Xbox Cloud Gaming posted an article to CNCF about their experience with LinkerD during their cloud native journey
https://www.cncf.io/blog/2022/05/10/...

LTTP but I welcome discussions of devops. I and my team are trying to just get to the point where were following some best practices, e.g., using classes and inheritance/composition at all rather than writing lots of redundant code for procedural execution, docstrings for functions and classes, how to even use git correctly (don't get me started on commit messages like "Fixed something" or "Maybe git will like this finally"). But I would love to get us doing at least a little devops so we can do CI/CD-type commit/build processes. We currently manually compile desktop apps from our code and it's not awesome in any way.

Ive spent the last month learning (relearning really) Unreal Engine and Blueprints. Just working on a personal project for fun but I have to say it's been extremely enjoyable using Blueprints to create game logic. Epic has done an amazing job converting C++ into a fully capable visual scripting language.

Been a while since I dipped in. I'm writing an MSc level course on ML-Ops, data engineering and high throughput data analysis platforms (hadoop, storm, luigi, CWL etc...). I'm just compiling some options for the reading list. If anyone here does this kind of stuff is there any material out there you think is especially good? Could be a book, a website, an unusually well written manual...

muraii wrote:

LTTP but I welcome discussions of devops. I and my team are trying to just get to the point where were following some best practices, e.g., using classes and inheritance/composition at all rather than writing lots of redundant code for procedural execution, docstrings for functions and classes, how to even use git correctly (don't get me started on commit messages like "Fixed something" or "Maybe git will like this finally"). But I would love to get us doing at least a little devops so we can do CI/CD-type commit/build processes. We currently manually compile desktop apps from our code and it's not awesome in any way.

Do you work at my last place of employment(maybe my last programming gig...)? Owner forbid the use of inheritance and were talking about moving to git over a year before I left(There is/was version control of another variant). Also no CI by the time I had left.

I feel weird in this thread. I picked up my software degree in my 30's, did a couple 6 month internships, even had a full time offer for one but flubbed the technical interview(twice!) Held a programmer position full time for just under a year, then covid, then I tried to WFH with 3 kids and ended up bailing completely. Honestly I haven't written an honest line of code in 1.5 years.

Every position seemed to be grooming me towards Devops though. I think it's time for a personal project to see where my future may lie.

Druidpeak wrote:

Owner forbid the use of inheritance

Curious to hear that story. I can understand encouraging composition over inheritance but outright forbidding it?

pandasuit wrote:
Druidpeak wrote:

Owner forbid the use of inheritance

Curious to hear that story. I can understand encouraging composition over inheritance but outright forbidding it?

Not sure. I overheard the manager go to bat for the programmers, but it was immediately clear this was an old battle and the higher ups weren't going to budge.

What language was it where inheritance was forbidden?

C#

Hey boss, I understand you're having inheritance problems...

IMAGE(https://i.imgur.com/gAi9zfg.jpg)

Yup, they were not from a programming background. Had certainly put a lot of time into development over the years though. Easy for me to bash, yet they have been quite successful. If it ain't broke, don't allow it to have children?