Back in 2011 when I was running multiple clusters of big (t2.xlarge style) instances I was seeing massive but seemingly random slow-down issues. I'd naively assumed it was 'the netflix neighbours', and/or STW GC issues with the JVM we were using, mostly because the cpu credit computation wasn't well known at the time.
I wondered if it would even be worth considering. I'm glad this comparison is out there; I'll stick with Vultr for VPS, and my own bare metal when I need more power/storage.
I have mixed feeling about Vultr. I dont like the "we've restarted your instance because hardware was having issues". Lately, the performance on some of my instances has degraded significantly. This was resolved by moving to a different zone. I dont expect a $5 VPS to be speedy, but there's minim performance that I expect. I wouldn't necessarily steer anyone away from Vultr, but I would recommend a thorough investigation of options.
Hmm, I've never experienced that on any of my instances, apart from the Minecraft server that I run for my niece and her friends. It got overloaded once when a bunch of players signed on at the same time, and I just bumped up to the next highest slot. That was my fault, since I was cheaping out trying to run it at the $5 tier; Minecraft is a memory hog even if you tweak the settings. Anecdotal, of course.
I have had serious performance issues at Digital Ocean, which is one reason I moved away from them and to Vultr (the main reason being Vultr's support for uploading your own custom ISO; OpenBSD in the cloud is cool). One example of many: I ran a custom website on DO for a client for a couple of years, but it was constantly going down despite the logs showing no significant traffic spikes and no Apache or PHP errors. I moved it to a shared hosting plan with my reseller provider and it has been up 100% since, not counting planned downtime.
All of that said, if I ever start experiencing the issues you're describing, I'll move on in a heartbeat. I don't do anything mission critical on Vultr, it's mostly for testbed stuff and that Minecraft server, nothing I can't move to another provider with ease.
Not surprised, Lightsail servers are just t2 burst instances rebranded.
Honestly, even using EC2 c4 (compute) or r3 (memory) instances it is hard to match the benchmarks of larger DigitalOcean droplets in terms of I/O. EBS is just really hard to get performance out of unless you use local disks and storage optimized instances.
Yep, and that's the point of storage-optimized :) amazon will happily sell you whatever all day with no claims about fitness for any specific purpose. It typically really pays off to benchmark your actual workloads on different instance types to figure out the best RPS/$ or whatever your metric is.
Does the lightsail instances have a hard CPU credit limit as well? The "let's turn into a snail in the middle of a kernel update and shaft everything" one.
Wasn't that the finding of the article? That once the load required enough of the CPU over enough long time, CPU credits ran out and the CPU was throttled?
There's CPU throttled and then there's the 'completely undocumented useless wall of pain' I find with T2 instances which is a bit after that. A throttled box is usually just about responsive but there is a point of no return. This might be a side effect of the throttling however.
> It took only 2 hours after setup and an average 25% cpu usage for the cpu steal to start climbing, just a few percentage points at first, but more than 80% after one more hour.
Sounds like CPU throttling is not an on/off thing but kicks in gradually.
Sounds like they throttle you, then only give you excess resources after a certain point. Kinda kills the point of a server if it grinds to a near halt, the initial throttling is workable.
Sort of stupid to price it out there like that then. To me it just seems a cheap way to mess with the lowend market. Try to get some of those people using AWS, so you can hook them into your expensive offerings.
Eh, Lightsail is very stingy compared to competitors in the lowend market, and the CPUs & SSD space don't linearly scale which will deter many who forsee the need to scale their container as their needs grow.
lowendtalk.com can show ya what Lightsail is competing against, IMO more BW, SSD Space and less aggressive throttling are needed if they are to compete with even the higher end of that market, it comes off as a stingier knockoff of Digital Ocean & Linode.
It could alternatively be framed that it benefits users who could learn some AWS through using their low-end stuff. (I guess AWS skills are probably desirable in the job market).
The obvious followup question: is lightsail the same interface / setup as normal AWS so that this would indeed work? or is it some sort of vastly simplified separate thing?
Lightsail does require having an AWS account, and I do believe you still have to go through the same setup (verification, sign up with a credit card). But the interface to create machines is vastly simplified.
You choose either the base OS, or an app stack (like Nodejs, or LAMP), then the price you want to pay, then name the machine.
If the article's evidence is the reality of Lightsail, it's clearly not a production platform. Amazon does state that the intent is really for developers to quickly spin up instances in which to experiment, and being able to use Amazon's unique Linux image that contains the latest AWS hooks makes for a good way to develop apps for later deployment to a fully-enabled AWS instance.
It's $10 a month. They do offer bigger instances. The t2.micro instance this maps to gives you 10% of a CPU and bursts to more. It's not much, but it is predictable and, crucially, not a function of the amount of oversubscription by the vendor or CPU use by your neighbours, both of which are typically unknown variables.
This would be nice, but actually getting those services set up is currently opaque. I have a service request in asking how to actually tie one of these instances to a load balancer and it is just sitting there unanswered.
Are network transfers from EC2 to Lightsail free? (in the same AZ)
If so, you could route all your outbound EC2 internet traffic through a pool of Lightsail servers. You're right that it's an order of magnitude cheaper. 1TB is $5 through Lightsail v.s. $90 through regular EC2.
As long as you use private IP addresses, AWS-AWS traffic is free[0]. However: Lightsail instances don't show up as EC2 instances nor can you add them to a VPC. Meaning that your AWS resources need to be in EC2 classic in order to have free transfer between them. And even then, you would need to do it with VPN tunnels to get there. Still, could be worth it.
Any idea if S3 => Lightsail bandwidth is free? If so then that might be an interesting play.
It'd be relatively straightforward to put a proxy in front of S3. It'd end up be 19x cheaper (pool of 1TB @ $5 Lightsail servers) for bandwidth on the low end and 5.5x cheaper on the high end (pool of 5TB @ $80 Lightsail servers). Compare that to $90 / TB for standard S3 outbound to internet pricing!
Its really hard to beat the price/performance of Linode. The closest match might be a dedicated machine from someplace like liquid, even then I wonder...
My only problems with Linode thus-far have been KVM related bugs. The older Xen type instances never had problems at all, at the second to largest size of Linode though I've run into a series of KVM bugs that cause the VM to lockup entirely or pause for seconds to minutes. Its frustrating, but they have been incredibly helpful in fixing the problems as they come up. I have a hard time everytime it happens of considering moving... and I always end up staying with them.
Good call ! I think the value of lightsail is more of what you get along with it (S3, other products and a path to scale up) than the vps itself (which can be had by less than both aws and DO are charging on alternative providers). DO is hard to beat on iops but it has been years they are asked to release a loadbalancer or basic stuff you can horizontally scale your apps. Linode has that but their VMs seems to "age" faster than DO (in terms of losing performance after some time due to neighbours).
I appreciated the mention of the OVH option, as that's the first vendor I've seen significantly below $5/mo. I'm currently using a reserved t2.micro at $2/mo, but it's nice to see there are other options if I need a little more power for the same money.
I am currently running all my browsing through a self-hosted VPN on OVH's cheapest SSD (GBP £1.50/mo on Black Friday), and I'm happy so far. Bandwidth is limited to 100Mbps, but my home connection is 55Mbps, so no problem.
In fact, one thing this highlighted for me is that Linode disables AppArmor on their kernels (running Ubuntu 16.10) — which seems like a slightly shitty thing to do, especially for a company with a somewhat flaky security history — whereas OVH doesn't.
Fortunately, Linode supports running custom kernels like those provided by the distro. That's how we run our Linode slices with the stock kernels from Ubuntu.
I've worked with linode, do, and aws over the years. I'll skip do as they're fairly similar to linode but I haven't used them enough to have explicit examples, and will therefore compare linode and aws.
Off the bat, you get a lot more bang for your buck at linode, in terms of compute and storage. If you're running a single server, don't care too much about what's on it or having many nines of uptime - go for linode. I still use them for hosting a few dinky personal projects.
If you're looking to do more though, say something involving hundreds of instances and thousands of interactive requests per second, and every second of downtime means real pain, then raw compute is far from your only consideration.
When we had dire issues with jitter on the xen packet scheduler with linode which was causing memcached requests to get stuck linode support really tried, but drew a blank - we had to tell them the solution and beg them to implement it, which they did - but it took a month, and it cost customer confidence, revenue, my sanity. It's little things like being limited to 50Mb/s node interconnect that make life unnecessarily hard. I absolutely understand why that was the case there, and was very grateful to the folks there tweaking our XPS settings so we wouldn't end up with our packets essentially being quantised, but it was the kind of thing that you really don't want to have to deal with - like moving into a house (admittedly running a heavy industry out of it), and then finding that the windows emit gamma rays on Tuesday when it rains - it took an absurd amount of work to figure out what was going on.
We moved to AWS in the wake of this and other similar issues. Big. Bureaucratic. Expensive. Reliable. Knowledgable. Cheaper than failure.
While since moving to aws we saw occasional and sometimes substantial weirdness within amazon's stack, they would always escalate and resolve major issues very swiftly. The longest we ever waited was a week, on something that required a global change, as we'd discovered a major problem with failure behaviour of one of their beta services. They talked to us every day. The management tools and api are comprehensive, the ecosystem encompasses most services you might want but doesn't get in the way of rolling your own, and they're remarkably flexible in what they'll reconfigure for you if you ask with a good reason. Paying £100k+ a month probably didn't hurt either.
The long and the very short is that you aren't purely paying for hardware when renting compute, you're paying for everything surrounding it. AWS cost more, but for the peace of mind of being able to straddle AZs easily and to know that if something on their end goes bang they'll fix it quick it's worth it.
We managed 97% uptime on linode over two years, five nines on aws.
Oh, and as an aside - when you're selling a SaaS solution, your customers have a lot more confidence when you say the basement iron is Amazon, rather than "sorry, can you repeat that? Lipode? Lytrode?".
I really don't want to bash the linode guys - Chris was really helpful in the early days, and the personal touch was awesome, I saw a lot in their "DNA" that was like ours, and as above, I do still use their service - but AWS is kinda in a league of its own when it comes to provisioned compute services.
Thanks for the writeup. I've never used Linode, but I recognized lots of echoes of my own experience with AWS, with the big point being something that isn't always given the importance that I think it deserves - when you get to larger deployments, you are much better off being a comparatively small fish. We started on colocated hardware, set up some new services in AWS, and scaled them up; we pushed the components hard in various weird ways and actually tickled some bugs in the c4 instance class when they first came out, but before I was able to figure out repro steps my phone was ringing and there was an engineer on the other side, and that's just because they wanted to figure the problem out - we did not, and still don't, have a paid support plan. But by virtue of being likely to affect other users, if we'd managed to trigger it, it was worth solving for them. I've certainly never had anything like an experience where I was trying to get them to fix a problem; any issues simply have to be fixed, as they will invariably end up affecting (many) others.
More recently, as we outgrew our original colo hardware, we got set up on OVH dedicated servers, which was back-of-envelope cheaper. Between delays in getting hardware online and managing to trigger the DDOS mitigation in a couple different scenarios, with the debugging and frustration you can imagine that entailed, I don't know if that's turned out to be the case. But when you're dealing with AWS (and I imagine the same should be true of GCE & Azure), it's really simple - if you have to ask, you're nothing special. You're not going to exhaust the available hardware. You will never wait N weeks to get another rack operational.
I can't really say too much other than they looked to be more "here's a box", like linode, rather than aws who are far more in the full vertical integration business. Only used them to host a few sites years back, and never pushed anything hard enough to break it.
My general advice is for personal and non-critical/demanding stuff go for bang for buck, but for critical stuff (I.e. Ecommerce) you can't understate the value of a robust and effective support infrastructure.
I remember the really bad old days when I'd have to drive to downtown Chicago at three in the morning to fuck with a physical server in a rack in a colo. So glad to be able to offload that end of things.
This is not an acceptable time format.
Back in 2011 when I was running multiple clusters of big (t2.xlarge style) instances I was seeing massive but seemingly random slow-down issues. I'd naively assumed it was 'the netflix neighbours', and/or STW GC issues with the JVM we were using, mostly because the cpu credit computation wasn't well known at the time.