This is too metaphorical, but, still, basically correct. Nice to see that.
Essentially, in the latent / embedding / semantic space, "seahorse emoji" is something that is highly probable. Actually, more accurately, since LLMs aren't actually statistical or probabilistic in any serious sense, "seahorse emoji", after tokenization and embedding, is very close to the learned manifold, and other semantic embeddings involving related emoji are very close to this "seahorse emoji" tokenization embedding.
An LLM has to work from this "seahorse emoji" tokenization embedding position, but can only make outputs through the tokenizer, which can't accurately encode "seahorse emoji" in the first place. So, you get a bunch of outputs that are semantically closest to (but still far from) a (theoretical) seahorse emoji. Then, on recursive application, since these outputs are now far enough from the the sort of root / foundational position on the manifold, the algorithm probably is doing something like an equivalent of a random walk on the manifold, staying close to wherever "seahorse emoji" landed, but never really converging, because the tokenization ensures that you can never really land back "close enough" to the base position.
I.e. IMO this is not as much a problem with (fixed) tokenization of the inputs, but moreso that tokenization of the outputs is fixed.
It is important to understand the difference between moral laws, which are immutable, and other laws in the Bible, often categorized as civil or ceremonial.
In modern legal theory, we have a similar distinction, which you may know from Legally Blonde: malum in se (that which is inherently wrong) and malum prohibitum (that which is wrong because a lawful authority prohibits it). Example: endangering the safety of others by driving too fast is wrong in itself, while breaking the speed limit is wrong because the town said so.
The former type (moral, and in se) doesn't depend on the existence of a government or system of laws and justice. The latter type certainly does, and the wrongness of those things 1) is changeable at the will of the governing authority, 2) applies only to those properly under the authority, and 3) expires when the governing structure expires.
Christians believe the ten commandments are moral law, whereas when we get into the ins and outs of the ancient Israel system of justice, described at length in Leviticus, eg, we are seeing one civilization's implementation of that moral law into civil law, coupled with religious law and ceremonies that symbolized and pointed to deeper realities.
When Jesus says he does not abolish but fulfills the law (and he means all of the law, not just some laws), he can't mean you are now free to murder. He means if you murdered someone, you may yet receive eternal life and not eternal death, because he can pay the penalty on your behalf. He means he lived a perfectly moral life and can impute that to you. That's fulfillment of moral law.
It also doesn't mean eating pork was immoral and now it's moral. That dietary restriction was a malum prohibitum component of the ceremonial cleanness symbolism that was meant to point the Israelites to the reality of what happens to a soul that consumes unholy things. It was never immoral, only prohibited. But the law is fulfilled in that the symbol is no longer required. Jesus institutes a new symbol that fixates on his own holiness and cleanness imputed to us as if by eating it (the Lord's Supper). The early Christian movement quickly incorporated non-Jews who had no heritage of living under the authority of the Israelite state, which was, by this time, already expired/expiring, many times over through successive conquests and occupations by various other governments. Neither the non-Jewish Christians nor the Jewish Christians needed to abide by those laws, though many did for a time, as it was an integral part of cultural and religious custom.
The relational model (and generally working at the level of sets/collections, instead of the level of individual values/objects) actually makes it easier to have this kind of incremental computation in a consistent way, I think.
There's a bunch of work being done on making relational systems work this way. Some interesting reading:
the 2000s were a lawless hellscape of chumba wumba, shrek and apple stores where you had to have core strength to consume the product. the average apple employee had to be resilient enough to withstand half a dozen yoga balls per shift fired at terminal velocity from the endless batallions of corpulent arse fitted to cloistered exurban trophy wives who only want and consume. during the holidays it was a bloodbath of fat slippery arses embiggened in so much wetzel pretzel butter from the seldom sanitized food court catapulting plastic denizens of futurism into unsuspecting children and macbooks alike. these corporate wardogs also had to be savvy ebough to explain in plain english that 2000 era maps would inevitably plunge the family truckster off a cliff if given half a chance, and that the iphone 1 had to be charged six times a day for the privilege of flexing on karen bakowdkis shotty latke recipe in real-ish time.
I had movers a couple months ago and tried to tip them. Note that I don't ever carry cash and am a proponent of abolishing the Federal Reserve. I asked them for their Ethereum or Bitcoin address, and they said, "neither of us don't have that, but we take cash." I calmly tried to explain to them the evils of the Federal Reserve for around five minutes, then began to set them up with their own cryptocurrency wallets on their phones. I then plugged in my Ledger Nano S and had them each read me their public keys, which I entered into Ledger Live and sent them each their five dollar tip. It was a simple process and did not take longer than thirty minutes. By the end of the exchange, they were both very happy with the arrangement and thanked me profusely for setting them up to use the future of currency, although I was still in the process of explaining its importance.
"Teach a man to fish," they say, and I believe I did it on that day. I encourage everyone else to do the same.
Yes. The CPU and GPU demand has nothing to do with it. The reason is the car industry.
For some reason in early 2020 all the car industry execs were convinced that people would buy dramatically fewer cars in 2020, due to pandemic crashing demand. Because they have a religious aversion to holding any stock they decided to shift the risk over to their suppliers, fucking said suppliers over, as the car industry normally does when they expect demand shifts. The thing that made this particular time special as opposed to business as usual is that the car execs all got it wrong, because people bought way more cars due to pandemic rather than less, due to moving out of cities and avoiding public transit. So they fucked over their suppliers a second time by demanding all those orders back.
Now, suppose you're a supplier of some sort of motor driver or power conversion chip (PMIC) in early 2020. You run 200 wafers per month through a fab running some early 2000s process. Half your yearly revenue is a customized part for a particular auto vendor. That vendor calls you up and tells you that they will not be paying you for any parts this year, and you can figure out what to do with them. You can't afford to run your production at half the revenue, so you're screwed. You call up your fab and ask if you can get out of that contract and pay a penalty for doing so, and you reduce your fab order to 100 wafers per month, so you can at least serve your other customers. The fab is annoyed but they put out an announcement that a slot is free, and another vendor making a PMIC for computer motherboards buys it, because they can use the extra capacity and expect increased demand for computers. So far so normal. One vendor screwed, but they'll manage, one fab slightly annoyed that they had to reduce throughput a tiny bit while they find a new buyer.
Then a few months later the car manufacturer calls you again and asks for their orders back, and more on top. You tell them to fuck off, because you can no longer manufacture it this year. They tell you they will pay literally anything because their production lines can't run without it because (for religious reasons) they have zero inventory buffers. So what do you do? You call up your fab and they say they can't help you, that slot is already gone. So you ask them to change which mask they use for the wafers you already have reserved, and instead of making your usual non-automotive products, you only make the customized chip for the automotive market. And then, because they screwed you over so badly, and you already lost lots of money and had to lay off staff due to the carmaker, you charge them 6x to 8x the price. All your other customers are now screwed, but you still come out barely ahead. Now, of course the customer not only asked for their old orders back, but more. So you call up all the other customers of the fab you use and ask them if they're willing to trade their fab slots for money. Some do, causing a shortage of whatever they make as well. Repeat this same story for literally every chipmaker that makes anything used by a car. This was the situation in January 2021. Then, several major fabs were destroyed (several in Texas, when the big freeze killed the air pumps keeping the cleanrooms sterile, and the water pipes in the walls of the buildings burst and contaminated other facilities, and one in Japan due to a fire) making the already bad problem worse. So there are several mechanisms that make part availability poor here:
1. The part you want is used in cars. Car manufacturers have locked in the following year or so of production, and "any amount extra you can make in that time" for a multiple of the normal price. Either you can't get the parts at all or you'll be paying a massive premium.
2. The part you want is not used in cars, but is made by someone who makes other parts on the same process that are used in cars. Your part has been deprioritized and will not be manufactured for months. Meanwhile stock runs out and those who hold any stock massively raise prices.
3. The part you want is not used in cars, and the manufacturer doesn't supply the car industry, but uses a process used by someone who does. Car IC suppliers have bought out their fab slots, so the part will not be manufactured for months.
4. The part you want is not used in cars, and doesn't share a process with parts that are. However, it's on the BOM of a popular product that uses such parts, and the manufacturer has seen what the market looks like and is stocking up for months ahead. Distributor inventory is therefore zero and new stock gets snapped up as soon as it shows up because a single missing part means you can't produce your product.
So here we are. Shameless plug - email me if you are screwed by this and need help getting your product re-engineered to the new reality. There's a handful of manufacturers, usually obscure companies in mainland China that only really sell to the internal market, that are much less affected. Some have drop-in replacement parts for things that are out of stock, others have functionally similar parts that can be used with minor design adaptation. I've been doing that kind of redesign work for customers this whole year. Don't email me if you work in/for the car industry. You guys poisoned the well for all of us so deal with it yourselves.
This is a very specific instance of a much more general problem.
A lot of private companies control exclusive access to something with a value that dwarfs what you pay for it. I pay nothing for access to Twitter; if I build a business or a social life on the platform, it becomes something I would pay thousands of dollars to prevent losing. I pay nothing for access to Facebook; the memories they store at this point in my life may be nearly priceless. I've paid a low triple digit sum for Blizzard games, yet the time and social investments I have made in those games make them a couple of orders of magnitude more valuable to me, now.
The problem is that since these companies control services so valuable to me, anyone who wishes to hurt me for any reason can do it through them. Since no one is paying them to defend me -- I'm certainly not -- they have no resources commensurate with the value of what they're defending.
The situation we're in now is one in which political thugs apply pressure to private companies to hurt individuals, in an attempt to chill free speech.
Free speech is expensive and valuable, and defending it from those who would wish to destroy it requires commensurate resources. We should not expect Blizzard to stand up to the Chinese government; that is the job of the Chinese people, of other goverments, of perhaps the whole world.
To my view, Blizzard is like a store clerk who gives up the store's money to a robber. It would be nice if he was a hero, but he's not equipped for it. Nobody is paying 7-11 to stand up to violent crime. The problem is too big and expensive to ask individuals to deal with. Society paying for police and courts is at least a response on the right scale.
The mechanisms we have for protecting individual rights are antiquaited, and need to be rethought to deal with the current situation. Perhaps a model like the unified response to patent trolls could work? I think, if we want free speech to exist in the current environment, it will have to be something that big.
This is the hack I use: once I’m ready to stop for the day, I figure out a solution (usually to a small bug) but I stop myself from coding it. The next day, the code just flows from my fingers to the keyboard the second I sit down. Works like a charm and I’m right back into it. Just delaying the gratification of seeing another bug crushed.
At least to some extent, this is a "Hegelian pendulum."
In the beginning there where black suited corporate drones. They coded black and white "calculation solutions" for the enterprising enterprise.
Then came colour. Gates wanted cheap computers on every desk. Jobs wanted colour computers for every artist. Mission. Meaning. This worked better, especially if you needed employees to invent things. Can't invent the iPhone wearing a suit writing business requiremnt compendiums.
Startups hired world changers and made a customer and emloyee happiness index. Google wrote "Don't be evil" on a chalkboard. Gmail was better than outlook. All was good.
Then some tension surfaced. Advertisers thought they would be happier tracking users. Users thought this was creepy. Making generals happy did not make employees happy. Someone erased "don't be evil" from the blackboard.
Back to square one.
Google (and "SV") have spent 2 decades telling everyone about their open and selfless ideologies. People bought it. They went to work for Google instead of a bank. Now, they use euphemisms and secrecy to avoid saying stuff that sounds bad.
Btw... It's interesting how iconic "don't be evil" was as a slogan. They couldn't live up to it, but that doesn't mean it didn't impact.
Yes there are licenses to ensure safety and compliance for plumbers, electricians, and hair stylists. And if I understand what I'm reading, some people here calling for similar "licensing" for anyone who wants to report the "news". While I'm in favor of the former, I am strictly and uncompromisingly against the latter.
Freedom of the press, the freedom to report on the news, opine on the news, and editorialize the news, in my mind is sacrosanct. "Incorrectly" reporting the news is not false advertising, and I would strongly hope that any attempt to license, monitor, or censor any news outlets (no matter how ragtag or unpopular) would be shot down hard by the 1st, barring the well established limits around direct incitement of violence.
Ingredient lists are a public safety measure. People with food allergies eat a mislabeled product and they die. People with an allergy to Fox news can change the channel and listen to CNN if they so choose. There isn't a 1st amendment right to sell someone a product (like a can of soda) and lie to them about what is in it.
I can debate the merits of any article from the New York Times, the Washington Post, or even Breitbart. I can debate how much an article in any of those publications seek to neutrally inform, or seeks to present a specific viewpoint, or seeks to outright persuade its readers of what to think. Reasonable people will disagree emphatically in such a debate.
But no reasonable person can disagree that the can of Coke Zero sitting next to me contains; Carbonated water, caramel color, phosphoric acid, aspartame, potassium benzoate, natural flavors, potassium citrate, acesulfame potassium, and caffeine.
Watch 5 minutes of news coverage of Comey's book on each of the major networks. Now tell me which were news and which were editorial. Spoiler alert: It's a rorschach test. Even objective news is not free of characterization and choice of diction which colors the facts being reported. Newsrooms have editors for a reason. Even the choice of which facts to report and which not is an editorial decision which must be made when reporting the news.
So I submit there can be no news that is entirely free and devoid of editorial. To report on Comey's book, you would have to sit in front of the camera and read it from cover to cover in a monotone voice without inflection or facial tic. And even that itself would be a type of performance art with its own editorial value.
For all the claims that fake news is "killing democracy" or "dangerous to democracy" I think the one thing that truly can kill a democracy is violating the 1st amendment and trying to establish some government censor of newscasts, podcasts, books, or vlogs because you think the message is wrong, misleading, dangerous, offensive, propaganda.
Based on the first-hand statements from Obama's campaign manager [1], I don't think we can characterize how they got the data as licit. It was, at the very least, wink-wink-nudge-nudge under the table. Facebook either really did know they could do that and didn't care, or their surprise was genuine, but they didn't stop it. Nor did the campaign have permission from everyone. Nor were they any more subtle as to how they used it.
To the extent that there was no deception in the Obama's campaign's usage, it was because there was no communication at all. This is not an improvement! And they got more data from this "no communication at all" than CA did.
Yes. Everyone was doing it. I totally agree we have a systematic problem here. But everyone was doing it and we have a systematic problem here works in all directions at once. Trump can't be singled out.
In fact, Trump may not even have used the data significantly [2], which makes it even more blatently obvious to the rest of America and the world that it's not a matter of principle, it's all a matter of who did it and whether or not we can tar certain people with it.
The question I have, and rather a lot of watching Americans and even the world have, is: Does "Silicon Valley" care about our privacy, no matter who is violating it? Or is "Silicon Valley" only suddenly concerned because the wrong people may tangentially have gotten something out of it, but when the "right people" do it Silicon Valley will do everything in its still-considerable power to cover for them?
One answer leads to beginning to recover the respect the public even quite recently had for Silicon Valley, but may require a lot of people in Silicon Valley to admit that, yes, "their guys" have been abusive and some corrective actions may need to be taken. (And for quite a lot of people, not just an ambient "yeah, some other people need to do something but not really me" but you, you dear reader, need to understand that abuses occurred and stop covering for them because they were done by the "right person".)
The other answer leads to a world in which this week was merely the warning shot in a war that Silicon Valley is basically structurally doomed to lose [3], no matter how invincible it may feel right now, and proving to the world at large that all their fears about Silicon Valley are completely, 100% justified. Which is it going to be? The hard way that's right in the long term, or the short-term psychologically satisfying answer that leads to doom in the long term?
[3]: Vast, vast quantities of Silicon Valley power and money are tied up in the premise that even now only a vanishing fraction of their market power has been exploited, and the best days are still ahead. If America and the world at large (the vast, vast bulk of which is not Silicon Valley liberal!) decide that Silicon Valley is the enemy, then all the stock valuations for things like Google and Facebook will more-or-less turn on a dime from "Companies that have immense growth potential in front of them" to "Companies very likely to be in short-term decline". The stock market carnage and loss in power and prestige that will go with that will be historic. Silicon Valley would be well advised not to play with this fire.
Something I think most Westerners who in liberal democracies don't understand is that the notion of "corruption" doesn't really exist in the rest of the world. At least not in the sense of being an immoral deviation from the norm.
What we think of as "corruption" would be better be termed "patronage", and it is the natural and default mode of political behaviour for human beings. It is impersonal, merit and market based systems that are the aberration, historically speaking. And, in the absence of strong institutions preventing it, societies inevitably revert to being patronage-based.
For example, imagine you are interviewing two people for a job. One is an extremely well-qualified and capable stranger. The other is a mediocre candidate, but is the son of a friend of your uncle. Who do you give the job to? For many people in the world, the idea that you would even consider not giving the job to the son of your uncle's friend is heresy. And they would not consider that attitude in any sense immoral or improper. Indeed, being in a position to help yourself and your acquaintances, and instead choosing to assist a stranger would instead be the perverse choice.
In a wider sense, within patrimonial societies there is never any question of ending patronage altogether. Instead, elite patronage networks compete with each other, and "anti-corruption" drives are about the dominant patronage-network cementing power and punishing another. We often see this in countries where every change in the government is accompanied by the prosecution of the old leaders for corruption by the new, only for the new themselves to be prosecuted a few years later upon leaving office.
Xi's efforts might be presented as being "anti-corruption" in nature, but they are very much about securing the position of himself and his patronage network. If he was serious about stopping corruption altogether, he would be concentrating on reforming China's institutions to make them more transparent, more merit-based and less susceptible to centralised political control. Instead, he's removing obstacles to himself and his patrons remaining in power indefinitely.
I don't go too crazy with things. But I have the caps > ctrl+esc, shifts > parens when pressed alone, and I have my backtick/underscore key swapped so a single backtick press gives me an underscore, and I do shift+hyphen to get a backtick (which I need far less frequently than I need underscores). Sometimes, when I feel like going really crazy, I swap my numbers and symbols so I get symbols as unshifted keypresses and numbers with shifted keypresses. But that sometimes gets confusing to years of muscle memory.
EDIT2: I also have my pipe symbol & backslash swapped (mostly for Elixir, where pipes are more common than in, say, Python). You can setup your own private.xml with Karabiner to do that like so: https://gist.github.com/bobwaycott/d3a52718927519f3a11fbff8b...
Agreed. The venn diagram of people that can afford the pro and those that care about emojis is likely rather slim.
However, watching the photoshop demo I could see the value. Honestly though, I think an add on accessory the size of a trackpad would be better. Unfortunately, that would sell in numbers so small that software support would be non-existent. So, this is a compromise solution that doesn't have Apple's usual boldness to it, rather a lacklustre add on that will deliver lacklustre results (to both sales and usefulness).
Essentially, in the latent / embedding / semantic space, "seahorse emoji" is something that is highly probable. Actually, more accurately, since LLMs aren't actually statistical or probabilistic in any serious sense, "seahorse emoji", after tokenization and embedding, is very close to the learned manifold, and other semantic embeddings involving related emoji are very close to this "seahorse emoji" tokenization embedding.
An LLM has to work from this "seahorse emoji" tokenization embedding position, but can only make outputs through the tokenizer, which can't accurately encode "seahorse emoji" in the first place. So, you get a bunch of outputs that are semantically closest to (but still far from) a (theoretical) seahorse emoji. Then, on recursive application, since these outputs are now far enough from the the sort of root / foundational position on the manifold, the algorithm probably is doing something like an equivalent of a random walk on the manifold, staying close to wherever "seahorse emoji" landed, but never really converging, because the tokenization ensures that you can never really land back "close enough" to the base position.
I.e. IMO this is not as much a problem with (fixed) tokenization of the inputs, but moreso that tokenization of the outputs is fixed.