hachyderm.io is one of the many independent Mastodon servers you can use to participate in the fediverse.
Hachyderm is a safe space, LGBTQIA+ and BLM, primarily comprised of tech industry professionals world wide. Note that many non-user account types have restrictions - please see our About page.

Administered by:

Server stats:

9.9K
active users

An enthusiastic YES to this from @henrikjernevad:
mastodon.social/@henrikjerneva

I recognize every single one of those examples from the post’s intro. And I recognize the pattern: developers imitating tech giants on small projects where those giant-scale approaches make no sense at all.

I like Henrik’s question: does it scale •down•? Does it scale down to the size of our actual team, our actual project?

1/

MastodonHenrik Jernevad (@henrikjernevad@mastodon.social)Why are people so obsessed with scaling? And how come people never seem to worry about whether technology scales *down* to a low number? https://henko.net/blog/does-this-scale-down/ #programming #softwaredevelopment #softwarearchitecture

A lot of my own software work has been for very small companies: arts nonprofits, early startups, one-person bootstraps, community orgs that are and always will be small.

These orgs have problems that are very, very different from the problems that, say, Google has. There’s no sense in them inheriting the miserable tech tradeoffs a giant corp faces! Enjoy being small! Do it the simple way! Stay flexible!

2/

You’d think that “it has to scale to 100 million users!!” is a pressure that comes from over-eager owners counting on massive growth, but in my experience, misapplying the patterns of tech giants to tiny orgs is a mistake that usually comes from us developers. We like shiny things. We over-scale either because the tech seems cool, or just because we feel like we’re •supposed• to.

But: What problem does the tool solve? Do we have that problem?

3/

Paul Cantrell

One of the problems that @henrikjernevad doesn’t get into in the blog post is that scalable code, like performant code, tends to be •brittle•: larger, more complex, more difficult to modify.

If you have only a handful of users, you have the incredible luxury of being able to write non-performant, non-scalable code that allows for •revision•. You can fiddle with design! You can learn from your mistakes and adjust quickly! It’s an •incredible• competitive advantage.

4/

It sucks being Google: if you so much as fart, you have a million users overnight. The second your product goes out the door, it’s half-calcified. Human-centered iterative design process? Forget about it!

Apple generally solves this by incubating things in secret forever. Google and Amazon solve it by training users to live with pervasive UX problems. If you’re huge, there is no good answer.

If you’re small, enjoy your incredible leg up: you have a chance to build good things fast!

5/

If you are tiny and looking to stay tiny…you have my heartfelt blessing.

If you are tiny and looking to grow, keep it simple, focus on building something great, and hope that one day you are •lucky• enough to have so many users that you need to rearchitect for scalability.

Honestly, more often than not, that moment of needing to rearchitect for scalability is a sign of •success•: somebody chose the tools that were appropriate to being small, and they succeeded.

/end

Consider: the cost of building supposedly scalable code and getting it wrong and having to redo it when scale actually hits is generally •higher• than the cost of keeping it simple and then having to redo it when scale hits.

@inthehands @henrikjernevad Yes! Flexibility is a trade off with stability/lack of brittleness.

@superflippy @inthehands My experience also suggests that the over-application of internet-scale technology comes from developers. To make it worse, I think a lot of it is doing what one think is needed to scale, without actually testing it, and likely not actually achieving it.

@inthehands wisdom coming out of Google decades ago: what works with ten users is so different from what you need with a few orders of magnitude more that you'll need to rewrite anyway

@inthehands it's too long ago to be able to dig out the slide deck with that any longer

@inthehands what I do remember is the aha moment I had in the big data/ nosql/ web scale years when @couchdb put scaling not only up but also down front and center.

@inthehands research.google.com/people/jef ... Found it. The design for growth, but not to scale infinitely and the 'numbers everyone should know' stuck in my head over the years

@mainec
Thanks, a very interesting deck!

The slide your remembered seems like really good advice: consider 5x growth, maybe 50x. Don’t plan for >100x growth; no matter how much you plan, that’s going to be a rethink and rewrite.

@inthehands Complexity is easier to add than remove. Simplicity lets you see more clearly where complexity can be added. I feel like most scalability concerns are wishful thinking and premature optimization, more YAGNI than reality. Working with a lot of legacy code, some of the most irritating and complex code is leftover optimizations for limitations that no longer exist. Add Google-scale optimizations when you have Google-scale money and Google-scale problems

@inthehands Very true. Main thing I advise Start-up clients to do is to focus on figuring out what is important now, and build the product in ways that it is kind of modular and easy to scale and extend in parts, when that time comes. This usually makes things less complex and encourages clean simple design patterns in all areas. For scaleups, I love the strangler pattern, to scale & rebuild the product bit by bit.

@rlcw All that. A question every startup should consider: “Would you rather (1) build something that runs slowly a year down the road and you have to upgrade/rework code, or (2) build the wrong thing — wrong product, wrong design — and find yourself unable to alter it before you run out of money?”

@inthehands very true. I'm currently working with a start-up founder who is taking that advice and it makes me very happy to work with him. We will build a solution for him that will be good enough for the customers, help him validate his core business model and quite a few of the processes do have a manual component in the background, that can be automated later, but in exchange gives him way more flexibility to adjust and change his business processes on the fly. I'm loving it.

@inthehands the example that springs to mind is Twitter. Twitter got to stupidly large whilst being written in Ruby, back when Ruby's only implementation was a painfully slow tree walking interpreter. They got much bigger, but Ruby and MySQL got them to the point where they mattered.

@pdcawley
Yes! Twitter and GitHub are the two examples I most often use.