This is a surprisingly unique product. AWS ECS and GKE both require some form of management of the underlying VM. A lot of that management is abstracted away, but not in the same way this is.
That being said, pricing [2] seems odd. At $0.0000125/GB_Second and $0.001/Core_Second, lets say you want to replicate an Azure A3 instance (4core/7gb/$130). That would cost... over $10,000/mo. Is my math right on this? It can't be.
For one-off jobs, maybe this makes sense, but as the backbone of a Kubernetes pool or something I'm not so sure.
But Hyper.sh is still competitive, I think, even though it's super young. They already have a high level AWS Lambda alternative (Hyper Func) that uses containers and is fully language agnostic, and they have reasonable pricing-per-minute figured out.
They also have open sourced their tech [1], so for certain companies they probably are compelling.
Perhaps they should provide the kind of ease-of-use on top of the stuff they already provide that will cut open a steady market for them, like a Heroku but with this underlying tech and thus more dynamic on scaling and flexible on tech stack.
For small teams like the one I work in, and for solo work I have dabbled a bit in, I can imagine the following thought process if I have a git repo and sit down to think about how to best deploy it.
- I start researching how to deploy this, but I don't care about optimizing for a lot of scale. I DO care a lot about iteration speed, integration with image registry + code repository + CI/testing services (ideally out of the box), and not making it hard to maintain/deploy/expand for the rest of the team (if an associate engineer can be brought up to speed on the infra and become fairly independent, that'd be wonderful).
- I probably care about burstable, pay-for-usage infra. But it'd be great if that was somewhat abstracted away. Perhaps even based on requests-per-minute, and I set up limits about how far it should scale. (This would immediately put this far and above Heroku -- good, cost-efficient auto scaling)
- I really care about not having to maintain my own DB cluster myself. And similarly for key/value storage and block-file-storage (like s3). And I'd absolutely love to not have to myself tweak and implement something common like cron or logging.
- And I'd probably care about integration with error-detection services and performance monitoring services.
- It would be absolutely amazing if there were default stacks/recipes with load balancers, so the common cases were easy (a common case like a standard RDBMS-backed JSON REST Api using cron and background jobs, or a static website (which basically covers all SPA-like frontends, if you add in a flexible reverse-proxy URL-rewriting solution))
Then, I could pick it up and be fine depending on it, since costs would scale down to my level (plus some premium I guess, but hey I'm even okay with Heroku for some things). But I could probably continue to stick with it for a long time. Inertia in ops means that if you get green field projects early and sustain them, my guess is that they stick around longer than they really even should -- but that's just an unverified guess on my part, idk.
ps -- the thing that really strongly drew me to hyper.sh was that I could abstract away the whole cluster behind `hyper`, like `docker` on my own computer. That was an amazing selling point, coming at after the popcorn machine of container management solutions with their very own intricate towers of complexity. It's what I like about sandstorm.io in part -- abstracting away a lot of complexity about hosting apps. But there's other complexity that could perhaps be eased or simplified, since running a software project is not simply the same as running an app on a computer, networking, load balancing, data security, backups and so on add up to a lot. So it's not enough that `hyper` abstracts away the data center, because there's scope for some more simplification for a certain slice of market: smaller teams who can't afford a full ops team, or teams with a small ops team who want to tackle a monstrous tech project.
Thanks for the detailed input. They are tremendously valuable.
Part of the reason we chose to open source is that we want the community to innovate. We are continuing to build the feature set, however I need to say that the workflow varies from app to app. Therefore, by providing the base building block and allowing others to create different solutions, we could enable more options to the market.
Yeah that makes sense. If there's enough of an enticement to adopt the infra service and you standardize the API at the bottom, any developments/contributions on top of it could co-exist and be more share-able. I'm reminded of the addon marketplace in Heroku. The underlying infra was consistent and general, which made third party plugins/addons/contributions easier (even to make money). Heroku ended up getting a lot of specialized support for platform customers that they probably couldn't have built out themselves.
I could totally see even seeding a possible 'marketplace' with your own services. Like perhaps Func as a service, powered by Hyper. And others also benefiting from the underlying infra being so simple. Why wouldn't a platform-provider use an infra service that's flexibly scalable on perf+price? They probably have the skillset on hand for that too and don't need any more finesse than already exists, beyond stability.
Anyway, that's just my own ramblings. Hope you're having fun with Hyper! It sure looks like a lot of love/tears went into it.
Yes, the billing is specifically targeted around per-second execution. The containers can start within a few seconds, they allow for customization of CPU cores and memory, and they allow you to focus entirely on your container without having to worry about any VM management. Traditional VM-based infrastructure is still the way to go for long-running applications. This just opens a new avenue into using containers in the cloud.
Off the top of my head, this could be used for backups, database migrations, auditing software, report generations, email blasts, CPU/memory intensive operations... The possibilities are endless because the execution layer is just a container, so you're not limited on what exactly is being run inside it.
Yeah, the examples they give during pricing definitely make it sounds like it's for one-off jobs or extremely short durations: "You create a container instance with a one core, two GB configuration 50 times daily during a month (30 days). The container duration of each instance is two minutes and 30 seconds."
I'm one of cofounders of Hyper. Thanks for mentioning us!
I think both ECS and GKE are valid options their segment. Things like ACI and Hyper.sh is more of a container-native IaaS. The on-demand event-driven compute infra is a perfect use case, but it also works for long-running workload (k8s pool), where the difference is that developers only need to consume the k8s api, not the software itself.
I don't think that unique really applies here - You can do the same with Google app engine standard runtime, or custom runtime if you want to use your own dockerfile.
https://cloud.google.com/appengine/
Last time I tried using GAE beta with custom Docker containers (±last year) it was an epic journey of pain and confusion. The docs are split-brain between standard and beta and the rapidly evolving APIs paired with equally rapid out of date (official) example projects is completely schizo. [edit: I should mention, though, I loved that they have those example projects at all! It was just frustrating to constantly be battling api obsolence in their projects.]
GAE is great, don't get me wrong. But it's so much more than just Dockerized app hosting, and it's not so great at doing just that. If hyper.sh has any half decent onboarding flow thought out, it should be a clear winner.
I feel that pain on the splitbrain docs. That's my number one complaint about their API's - I'm never sure if I'm on the newest. That said, there's a "familiarity" to them... once you figure out how things changed it's easy to quickly determine the age of a code sample.
You should try the custom runtime again! It's come a long way since Beta.
Good point about the "getting it" with their docs. I suppose there's always that ethereal point with new concepts... It's just frustrating when it takes so long to get there. And, somehow, once you're there, it becomes incredibly hard to empathize with people who don't "get it" yet. Consequently, docs suffer.
As the old joke goes: Once you understand monads, you immediately lose the ability to explain them to others.
Anyway, good to hear they've improved and stabilized.
It seems to me like it's supposed to be a more flexible, versatile equivalent of AWS Lambda: fired up for a quick operation based on some trigger, then immediately shut down. From that perspective the pricing doesn't seem too bad.
Yes. What hyper.sh is pitching for years is the "Container Native Cloud", and Azure ACI definitely shared the same idea. But will it become the future of cloud? I hope so.
Even with the fixed pricing data... it's still ~$360 a month, right?
So that's about 3x as much to not manage the VMs yourself in a cluster. For even a moderate cluster, that seems like it would rapidly stop making sense.
Depends. If it's a short job, then setting up a new VM for it or even planning where it should run might take a few engineer hours. And engineer hours are really expensive.
> An Azure Container Instance is a single container that starts in seconds and is billed by the second. ACI offer highly versatile sizing, allowing you to select the exact amount of memory separate from the exact count of vCPUs...
az container create -g aci_grp --name nginx --image library/nginx --ip-address public –cpu 2 --memory 10
There's also a k8s connector, promising faster spinup times:
So, are we just now using “serverless” to just mean “dynamically scalable” the same way that “cloud” used to?
Because, previously, “serverless” seemed to mean not needing to deal with anything lower-level infrastructure than function calls (that is, a higher level of abstraction than even a classical PaaS like GAE managed runtimes), while container hosting, dynamically scalable or not, is somewhere between classic IaaS and classic application-language PaaS.
We've updated the submission title from “Microsoft Azure launches serverless container instances” to that of the article, which doesn't mention serverless at all.
And both seems wrong to me. Serverless would be without server, not with an "intermittent" server. Serverless should design architectures without server processing per request (like jekyll compared to wordpress).
If you look at a reply to the comment you're replying to you'll see it previously in the title. It might have been posted between the change and your comment
While we've passed the point that this shows up in the timestamp, the message that indicated the change to the title was 1 minute before the one asking the question about where the term was used.
I'm a co-founder of a stealth-stage company that helps data analysts/data engineers build data pipelines. Every "task" that can be done in our framework is essentially just an image that can be reused over and over with different settings.
We deploy these tasks across Kubernetes clusters on AWS, GCP, and Azure.
Since these tasks are schedule irregularly and are short lived, we had to do a lot of work to dynamically scale the nodes up a head of their demand and down after, and we typically have to pay for at least 10 minutes of usage no matter how quickly the job finishes.
This "pay-by-the-second" will be a huge win for us. Most of our tasks deal with S3/Redshift or GCS/BigQuery, do we can't immediately use this. But as we onboard more clients working with Azure Storage/Data Lake/Data Warehouse I see some big operational gains for us.
Here's hoping we see similar developments across the other major cloud providers. Very impressed with Azure's development in the last 3 years!
Check out Hyper.sh too. It abstracts away the whole datacenter -- you can use `hyper` instead of `docker`, basically. You don't need to think about VMs ever as a concept, containers run directly on the hypervisor. And they have Hyper Func, an AWS Lambda-like alternative that uses images. And per second billing.
On the downsides, they're small and they have one data center, and they're not Microsoft. But their tech is open source.
We don't want to compete with the big providers, instead we open source the tech to enable more container-native clouds, where the world will become a seamless (portable) network for containers (different clouds are different ports with the same image spec and API).
What's really got me excited lately is the combination of Ansible (for dirty work) and container orchestration systems like Kubernetes/Rancher/etc (also, tools that go from one orchestrated host to many like dokku and flynn).
While I appreciate the competition from GCE and Azure, what I really want is a tool that will run in any one of their clouds, but offers the same ease-of-management, and lets me go from one cloud to another or to a private cloud without breaking a sweat. I want the competition to be 70% on price and 20% on added-management-value and 10% on bundled services.
Terraform is basically this tool, but I want an even easier interface, terraform still feels somewhat too specific to me -- I don't want to even have to write config or specify some "aws" adapter that will make my config work on some provider. I want instant, multi-cloud (possibly) heroku, using only the network, hard-drive-space, ram, and lxc "primitives".
Someone (maybe me if I ever find time) just needs to get to work making F/OSS versions of all the bundled tech (ex. blob storage, cloud function runners, dynamically configurable DNS resolvers, simple alerting, etc) that runs in a container, and then the question just becomes "where can I get the cheapest most performant VPS that will host my containers".
> Terraform is basically this tool, but I want an even easier interface, terraform still feels somewhat too specific to me -- I don't want to even have to write config or specify some "aws" adapter that will make my config work on some provider. I want instant, multi-cloud (possibly) heroku, using only the network, hard-drive-space, ram, and lxc "primitives".
This sounds very similar to some ideas I've been stewing on over the past few months. One of them being a multi-cloud terraform-like tool which abstracts away the low level details of which provider an instance is provisioned onto (whichever is currently cheapest). It could also automatically determine how firewall/security groups/networking should be configured.
If you happen to create two instances in two different providers that need to communicate privately over say port 443, the security groups would be updated automatically to account for this, bridging the two providers.
One major thing to consider in doing this is the bandwidth. If you look at the fine print in these providers, the outgoing bandwidth is where they really get you. So if your backup server is in a different provider than your database, you might see some hefty data transfer fees while performing a daily backup.
Just curious, what's your background? Are you an infrastructure type, or are you a general developer who just wants a tool like you've described? Sometimes I can't tell if this is something people really want or if I've just drank too much of the infrastructure automation koolaid.
tl;dr - "Fullstack" developer who does my own ops for my own projects which are very varied (I've done small-time deploys/management for projects in/with python, node, go, haskell, ruby, postgres, rethinkdb). I try to iterate my infra as well as my technology stacks to find the best way to do things.
Sorry for the delay -- I've definitely had this at the back of my mind too, but with every advance made by cloud providers, I wonder if I could ever possibly get into that fight, especially since I'm not an expert at any of the cloud computing platforms -- Hashicorp seems to be doing amazing work in this area also, field seems to be full.
My background is basically that I really enjoy computers/programming and like to be self sufficient so I prefer a shallow (working) knowledge of all parts of the stack. I maintain a few VPSes and use them to host n-tier (where n is usually 2) projects. I'm always interested in the best way to do stuff and I use new projects to try stuff out.
Ops is particularly interesting to me because I've gone through a lot of iterations of how I did ops for my apps -- from SSH-in-and-do-stuff -> use-fabric-to-ssh-in-and-do-stuff -> dockerize & ship containers -> dockerized + systemd -> dockerize & registry (thanks to gitlab) + systemd + ansible. I've also used dokku and of course it blew my mind (I never used/wanted to pay for heroku), so while I stayed relatively low level, I was cognizant of what was happening a few levels up.
I saw kubernetes a few years ago and wasn't sure whether I wanted to go with it, but I've taken a fresh look over the last week and think it might be time to give it a try.
Side (controversial?) note - I think the time where it's acceptable for junior/mid-level developers to only know one part of the stack is dead. The only thing keeping these roles possible are the fungibility requirements for large corporations. Smaller, hungrier (almost literally) corporations can't afford to have some developer that is only an expert at writing python but never works on frontend code or deals with the database.
Your story sounds so familiar, someone could read what you wrote above and think it's me.. though I like to think of myself as a generalist rather than "fullstack". I tend to believe "fullstack" is a marketing term for acquiring slave labor. You're right though, it's required at this point for smaller startups.
> I wonder if I could ever possibly get into that fight
I wonder this too. I've been using AWS for 4+ years at my day job(s). The main pain point for almost all users, including myself is the cost. I think a product/tool that helps companies save on cloud costs would be invaluable. The difficulty is convincing crusty old ops people that the product is something worth trying.
Anyway, after my current employer runs out of money, I'm going to do a startup of some sort. While one of my many idea was like the one we outlined above, I'm actually starting to shy away from the infra/op verticals lately.
My email's in my profile, if you ever want to bounce ideas.
Agree -- "Generalist" is the word I'm going to use now instead of fullstack -- that's the way better term, I put "fullstack" in terms because I dislike the term.
This is part of the vision that we have with CoreOS Tectonic. Checkout the Tectonic Installer which helps to install to various cloud platforms[1] using Terraform and then Tectonic[2] helps to automate operational toil like upgrades, etc.
Just a heads up -- some parts of the coreos site really make it seem like tectonic is only for enterprise -- it actually turned me off it while I was reading the documentation at https://coreos.com/kubernetes/docs/latest/
Right now my plan is to start with bare kubernetes just to cement understanding and then add rancher, before looking at any other higher level system (I mean there's stuff like helm out there, but it just seems to be too many levels above the ground for me at this point). I'm also using ansible right now (and relatively new to that too, but it's basically just fabric on steroids so I'm not too stressed about it), so I haven't started using terraform yet (but want to).
Hey thanks for the link! However I don't think Cloud66 is for me -- out of my price range for the small projects I build.
With respect, from what I can tell, most of the features of cloud66 are already included in Gitlab, and when I convert my current infrastructure to Kubernetes I can even start taking advantage of the auto-deploy-to-kubernetes features.
$49 / Month is a bit too much for me, but I don't think I'm the target anyway!
I'm also the kind that likes to run my own infrastructure and spend time on things I maybe shouldn't be spending time on :)
I hear what you're saying, but don't under-estimate the value of "someone else runs it and gives me an SLA", which is a large part of what public clouds are really selling. :)
the clear end goal here is that you have to deal with things like 'provisioning virtual disks' and 'doing ubuntu updates'. this whole virtualization thing received you of the burden of buying pci ethernet nics and rack mount brackets and provisioning cooling.
but really this whole business of writing chef recipes and provisioning harnesses is really the same kind stuff. it seems important because you can't run without it, and thats what your whole day is...but really its pretty secondary to what you're actually trying to accomplish (run a service)
its interesting to think about what that world might look like...someone is going to make something like that stick at some point. so...why are people provisioning their own containers/vms instead of using the higher level services right now?
For me at the very least it's cost -- I can purchase a cheap VPS and run a bunch of stuff on it, with full control for much cheaper than what an AWS EC2 instance costs monthly (with better specs).
The value provided by the services being managed is large, but honestly, for a lot of well-built infrastructure pieces, there is a lot of trust already for the services to not go down. Most startups/lifestyle companies/small businesses/whatever couldn't bring down a Postgres instance on a reasonably-provisioned machine if they tried (and the app is written with at least a smidgen of thought towards performance)
One of the aims of LinuxKit https://github.com/linuxkit/linuxkit is to show that you absolutely can do without the "doing ubuntu updates" and "writing chef recipes" stuff - you just write a single config file that specifies the whole OS config and the applications you want to run, test on your laptop, build in Ci and then deploy to cloud or bare metal, with just your service (or Docker/K8S for dynamic services).
The equivalent of AWS Lambda is Azure Functions. It does have the same memory limits as AWS (max 1.5 GB per instance), though you can run for up to 10 minutes.
The difference between it and other serverless offerings like Lambda and Functions is that you can run your jobs for upto 3 hours (after which we will restart it, not kill it, so if it is idempotent it will work ). It supports PowerShell/Python right now. Another core feature is being able to run scripts on your own VM from the Cloud called the Hybrid Worker. This has support both for Windows/Linux machines. https://docs.microsoft.com/en-us/azure/automation/automation...
(Disclosure: I am a member of the Azure Automation team)
This is only like AWS Lamda if a response time of a few seconds is acceptable to you. That's the time it takes to start up an instance according to the article.
FYI - AWS Lambda cold start times are also in the seconds [1]. I'm assuming Azure functions has the same limitation. After all, code has to be retrieved from storage over the network, environment set-up, and eventually, code needs to be run. This is a limitation of physics rather than software.
Azure Functions has support now for pre-compiled functions. Start-up is fast, under 2 seconds for me and simple Http triggers.
Would be interesting to see a mashup of Azure Functions and ACI. In other words, use a event-driven trigger to spin up a container instance so you wouldn't be limited by the "serverless programming model".
I'll give you an example - When I first discovered Lambda I had been working with containers for 6-8 months already and when I saw lambda it hit me that they just have container images capable of running python and node projects that this spin this up in and run the code we ask them to.
My mind immediately then went to our data pipeline that I'd like to run on a nightly schedule. This is typically done using a tool like Airflow or Luigi but I thought I could probably even just build the whole thing in lambda tasks that trigger via cron and SQS.
The reason we ultimately did not do that with Lambda was because it limited the amount of A. time and B. Compute resources one lambda task could use and we did not want to try to optimize for that when working with increasingly large amounts of data.
With something like this (ACI) I could actually do exactly this and even if its triggered by something like Airflow or Luigi I could run those on a much smaller instance and save the larger compute problems to being done on ACI in small spurts
As far as I understand, with services like AWS ECS you need to provision the infrastructure first and pay for its uptime, whereas this allows more ephemeral containers to be run with minimal setup, and you only pay for the compute time used
Would only be useful for short lived jobs, but a really nice idea none the less.
I don't want to spread FUD, but my understanding is that the container security model is not 100% and that's why people like AWS force your containers to run on EC2 instances.
The container security model will almost certainly improve in the future, but for now I'm only ok with other people in my same company sharing the kernel, not incentivised attackers.
[edit]
I'm going to unfud my comment. Some further reading makes me think maybe they spin up something like kvm containers and use a minimal distribution such that they can get to "seconds". If it were me, I'd have pre-running instances of the base image that were ready for a customer to attach and own.
Each container has hypervisor level isolation. We are not relying on kernel level isolation for security isolation between different user's containers.
I would like it if the major cloud providers implemented microsecond boot and teardown times for instances along with suitable pricing.
Unikernels and a whole zoo of other types of tiny operating systems would be enabled by this.
I'm not a fan of containers - I feel they are reimplementing much of the operating system infrastructure within the OS at the price of high and unnecessary complexity.
It's frustrating that cloud computing has so many benefits, but at another level we must wait and hope that Amazon Google and Microsoft are willing to implement new architectures such as microsecond level boot and teardown.
If you need service discovery, replicas, rolling deploys, etc. ACI probably isn't for you. Check some of the experimental work we released today connecting ACI with Kubernetes: https://github.com/Azure/aci-connector-k8s
Was looking to see if one of you guys turned up in this thread. "And how was the Deis team involved in this?" Didn't see any mention of you all in the article.
Thanks for showing up and weighing in on this! k8s connector looks really cool, is this a totally unique thing or are there anything comparable for ECS? I've never heard of a Kubernetes cluster with virtual nodes! Sounds like you could use this connector and potentially save yourself from ever needing to configure autoscaling in the Kube cluster.
I'm really curious how things are going at Microsoft for this incredibly productive team of people, from Deis, who have put out so much great software that has kept my attention. Hope that everything is great!
ACI lets you schedule individual containers (technically container groups, which are equivalent to K8s pods). Everything on top of that - scaling, service discovery, rolling upgrades, etc. - is up to you. That's why we think the Kubernetes connector [1] is interesting:
This doesn't seem cheap
>> £0.001 Core per second
That works out at over £2500 for a single core running for a month. And thats without memory costs etc...
That is very likely a display issue and they round up way way too much in that table. In the Pricing Example they use $0.0000125 per Core, which would make it ~33$ for a month for a single core, and double that if you include 1 GB of memory.
While it sounds cool i'm a bit dismayed by the naming choice as ACI in my head is short for the Application Container Image format as used by CoreOS and appc.
I find it funny to see containers landing in Windows. While I fully approve the containers on Linux, to me it looks like Windows does not really need them: it already has a stable notion of executable files with full binary compatibility. An old but gold EXE format is your container. Please excuse my probable naivety, but am I missing something?
Containers are not just about providing stable executable files, but consistent and reproducible environments. Besides, having a unified way of deploying both linux, windows or any other OS seems like a win-win since your backend (docker, or whatever container runtime you use) would remain the same.
Containers allow different EXEs to leverage different versions of the same DLL, much as the WinSxS folder does. It solves some problems that Windows previously solved, in new ways.
Also it prevents "server bloat", where a web or AD-LDAP server gets incidental utility roles attached, like running scripts or being a bastion/jump-box. You can't really RDP to the container, so you can't update DNS from there.
Containerization arose out of Linux namespaces. What you get is process segregation under the same kernel, like jails or chroots but more granual. So security, that's the answer.
The value-add here is not to be running cloud instances in containers. Rather it allows you to run containers and be billed by the second, which opens the doors to short-lived jobs running in containers on the cloud. It's closer to serverless platforms rather than VMs.
I built an internal tool where teammates submit some details and a fairly computationally heavy data processing task run and posts the summarised results back to a slack channel.
I use containers for this, because it's a really easy way to really quickly get access to compute instances (which vary in size from like 4gb ram to 100gb+ of ram depending on what the user submitted) really quickly and for really cheaply because I only pay for the time it actually runs, rather than renting out an EC2/VM instance all the time (which would either be underpowered, or end up costing too much), or build some frankenstein-ian thing to dynamically provision and spin up EC2 instances on demand, set them up and then shut them down/terminate them once they finish.
This is a surprisingly unique product. AWS ECS and GKE both require some form of management of the underlying VM. A lot of that management is abstracted away, but not in the same way this is.
That being said, pricing [2] seems odd. At $0.0000125/GB_Second and $0.001/Core_Second, lets say you want to replicate an Azure A3 instance (4core/7gb/$130). That would cost... over $10,000/mo. Is my math right on this? It can't be.
For one-off jobs, maybe this makes sense, but as the backbone of a Kubernetes pool or something I'm not so sure.
[1] https://hyper.sh/
[2] https://azure.microsoft.com/en-us/pricing/details/container-...