It's kind of sad systems research pretty much stopped at this point. I really was hoping that by 2024 I'd be running a distributed operating system where processes could be be freely migrated between my phone, desktop, laptop and NAS without too much of a hitch.
There is still a lot of research and innovation, but it doesn't always come in the way of completely new software projects. The cost of trying to build a new OS is simply massive. You need to be compatible with existing useful software if you want to do anything other than an appliance. Anything that provides a new paradigm shift that requires changing existing software has a huge slog ahead of it to make it successful. That said, there is tons of incremental progress in the operating systems.
I think a lot of folks have thought about the idea of a truly distributed operating system. I'm pretty sure existing operating systems will eventually evolve in that direction. You already see bits and pieces of it popping up.
While plenty of things have happened since that paper, I have this sinking feeling that he was right and we just stopped trying to really expand and explore what’s possible. But it may be more a matter of the state of academia than about the domain itself. It’s not like people were getting a bunch of conference invitations for GPGPU applications with ML until hype freight trains hit. This sobering reality of academic hegemony and grant chasing kept me from ever getting terribly interested in systems research unfortunately.
The research has not stalled at all, the incentive to make consumer devices has because of the capture by tech giants. If someone wanted to implement an interesting idea like this, they would be harshly judged for not being able to compete with pixels and iphones for the rest of the stack. See Rabbit for example.
What you are describing is the bread and butter of modern systems research and all the large cloud providers internally implement this kind of thing.
You should be looking at infrastructure related roles for this stuff; live migrations of VMs and containers are regularly done when you drain a VM or Kubernetes minion, for example.
For consumers, they mostly care about the user experience. Having software that syncs their contents to a server thus making it available to all devices has much lower overhead than trying to live migrate a process over unreliable networks.
This type of thing always seemed to be in the very cool but pointless basket, to me.
A lot of systems researchers were absolutely obsessed with clustering, network transparency, distributed systems, and viewed them as the pinnacle of the operating system. I never understood why. I completely understand the coolness factor mind you, I just never could see why it was so important that your server-laptop-phone-network system behaved as a single system.
I think a lot of effort was wasted chasing that dragon. Wasted is probably the wrong word because research into cool things is good and probably created useful things along the way. I don't feel there was ever enough justification put into it and it could possibly have been better spent though.
The alternative of having multi-system tools and programming models that allow you to manage multiple systems without having them appear as a single image at the lowest level didn't get much love from academia after TCP/IP, and was largely developed by industry.
We should have had a distributed OS like Amoeba/Plan9/Inferno/etc. allowing us to manage all our chosen set of devices using a single uniform interface i.e. "A Namespace" (in Plan9/Inferno speak). Such namespaces can themselves be connected into "Hierarchical Namespaces" and so on. This is a natural and easy way to let users keep their control over their devices while still being connected to the "Greater Internet".
But the Industry manipulated us into the Cloud model so that they could retain control and make money off of us. It was all Business to the detriment of a better User Experience via Technology.
> We should have had a distributed OS like Amoeba/Plan9/Inferno/etc. allowing us to manage all our chosen set of devices using a single uniform interface i.e. "A Namespace" (in Plan9/Inferno speak). Such namespaces can themselves be connected into "Hierarchical Namespaces" and so on. This is a natural and easy way to let users keep their control over their devices while still being connected to the "Greater Internet".
Why should we have?
> But the Industry manipulated us into the Cloud model so that they could retain control and make money off of us. It was all Business to the detriment of a better User Experience via Technology.
The choice was not cloud or distributed single system machine, they were and are orthogonal.
Because that is what an OS is supposed to do viz. provide a uniform interface and transparent access to various Compute, Storage and Network resources wherever they might be. A Distributed OS (https://en.wikipedia.org/wiki/Distributed_operating_system) is a natural extension to a Single Node OS. Note that we have in a sense realized our distributed OS in the IaaS and PaaS layers of a Cloud network. However they are done in such a manner as to take control away from us unless of course you use some open source offerings setting up which is much more complex than a distributed OS should be.
I don't find "we should do that because that's what we're supposed to do" a compelling argument. If you rephrase the assertion that just leads to a rephrased question, why is the OS supposed to do that?
There are many layers and components, and functionality can be implemented in many different ways in different levels of these things.
I am not sure what you are arguing here. The above definition + the previous link to distributed OS where the motivations for the same are listed establish the "reason". If you find other approaches more palatable that is fine (provided you know what you are doing) but the original blanket statement you made viz. "This type of thing always seemed to be in the very cool but pointless basket, to me ... I think a lot of effort was wasted chasing that dragon. Wasted is probably the wrong word because research into cool things is good and probably created useful things along the way. I don't feel there was ever enough justification put into it and it could possibly have been better spent though." is wrong.
> There are many layers and components, and functionality can be implemented in many different ways in different levels of these things.
All implementations are not equivalent w.r.t. all parameters; hence the various tradeoffs in competing designs.
Operating system purpose on some website is vague and not some canonical truth. It doesn't even necessarily say what you think it does, and doesn't justify itself if it did anyway.
> I am not sure what you are arguing here.
I'm not sure why you're replying then.
> All implementations are not equivalent w.r.t. all parameters; hence the various tradeoffs in competing designs.
Some Website? It's Wikipedia and curated; short of handing you a book (i recommend Tanenbaum) that is a good place to start from and then branch out of.
> It doesn't even necessarily say what you think it does,
That means you have not understood it.
> I'm not sure why you're replying then.
Because you made a untenable blanket statement that Distributed OS research was all wrong which cannot go unchallenged.
> Because you made a untenable blanket statement that Distributed OS research was all wrong which cannot go unchallenged.
You definitely didn't understand the first thing I wrote, or you're deliberately misrepresenting it because you're intellectually incapable of addressing it.
"Because you made a untenable blanket statement that Distributed OS research was all wrong which cannot go unchallenged."
That was not my direct quote, so you can dismount your high horse now.
And it's telling that responding to the question of why an OS should do these things, you finding a definition of an OS and asserting (falsely, I might add) that transparent distributed systems and only those meet that definition.
> That was not my direct quote, so you can dismount your high horse now.
You apparently can't follow a chain of comments in a thread nor understand what is written down. My comment pointing out yours is here - https://news.ycombinator.com/item?id=40784174
> And it's telling that responding to the question of why an OS should do these things, you finding a definition of an OS and asserting (falsely, I might add) that transparent distributed systems and only those meet that definition.
Again, you have not read/understood what has been written down nor the sources to which i linked to elaborate my points. And also i never said "that transparent distributed systems and only those meet that definition".
In my comment here https://news.ycombinator.com/item?id=40766810 i explicitly say "an OS is supposed to do viz. provide a uniform interface and transparent access to various Compute, Storage and Network resources wherever they might be." and then link to the distributed OS Wikipedia page for further details. If you had cared to even read that page you would have understood the key "Design Considerations" (see https://en.wikipedia.org/wiki/Distributed_operating_system#D...) driving the approach. It is the "User Experience" that makes a distributed OS approach so compelling; though its realization can be more complex/difficult compared to other approaches w.r.t. certain parameters eg. Scaling.
To summarize, i would advise you to read up on Distributed OS research before making silly blanket claims which can be easily disproved. Just because today the Cloud model has established itself as the industry standard doesn't mean that is the correct approach. You have to know both sides before you can argue for/against.
That's not the one I was responding to. I replied to the comment I was responding to, it's not that complicated.
> Again, you have not read/understood what has been written down nor the sources to which i linked to elaborate my points. And also i never said "that transparent distributed systems and only those meet that definition".
That was certainly the implication if you use that to answer the question, why must the OS do that.
You do understand how "The OS must do it because that's how I define the OS" does not actually say anything, right? Even if that is the definition of the OS (which it isn't), it just shifts the question to "why should the OS be defined that way?"
> That's not the one I was responding to. I replied to the comment I was responding to, it's not that complicated.
That is precisely the point; when something is referred to in a chain you have to look at where and not necessarily in front of your nose; it's not that complicated.
> That was certainly the implication if you use that to answer the question, why must the OS do that.
No, most certainly it is not when you are explicitly being pointed to resources where more details can be found. You are expected to do some self-study and not necessarily spoon fed when it comes to absolute basics like "why is a OS needed?".
> You do understand how "The OS must do it because that's how I define the OS" does not actually say anything, right? Even if that is the definition of the OS (which it isn't), it just shifts the question to "why should the OS be defined that way?"
You are either being intentionally obtuse or really are. The motivation for an OS is buried in its definition; this is CS101 and not something deep which you seem to think it is and i even linked you to the necessary definitions and mentioned terms like "Transparency, Resource Management, User Experience, etc." to highlight the essentials.
At this point i am can only conclude that you are repeating the same vague statements (they are not even an argument) over and over again without providing an iota of hard data (which i have done) because you really have no clue.
> running a distributed operating system where processes could be be freely migrated between my phone, desktop, laptop and NAS without too much of a hitch.
This was what i always wanted ever since i read Tanenbaum's "Modern Operating Systems" and in particular his "Amoeba distributed OS" - https://en.wikipedia.org/wiki/Amoeba_(operating_system) Also see Plan 9/Inferno from Bell Labs. But instead what we got (due to industry shenanigans) is this garbage/duct-tape of distribution gone crazy in the "Cloud".
If you really wanted to do this, it would be fairly trivial to implement with Erlang/Elixir. We have thr technology - just no motivation (i.e. no profit in it)
Hey! Don't let out the secrets :-) This is something i have been long thinking about (but have done nothing practical so far). The problem is how to bridge/shim between Erlang and those platforms which do not support it eg. Android (though some people seem to have done it - https://github.com/JeromeDeBretagne/erlanglauncher). Joe Armstrong actually called the Erlang/OTP System as a AOS (Application Operating System) in his paper i.e. it contains much of the functionality of a traditional OS but people seem to ignore it and insist on calling it "just another language".
IMO, you lose half the benefits by shimming OTP (genserver, hot-reload). Platform owners like Apple or Google could easily make Erlang processes first class citizens on their Unix(-like) platforms - one can already run full-fledged Erlang/OTP processes on Android (via Termux).
I imagine in an alternative universe were interoperability is more profitable than walled gardens, this technology exist and was standardized and not just a preserve of a few services on iDevices that can "hand-off" to each other (without process migration)
