Besides, you probably don't want "done" jobs in the same table as pending or retriable jobs - as you scale up, you likely want to archive them as it provides various operational advantages, at no cost.
Not false. Nobody would ever use BRIN for this. I'm talking about regular indexes, which do prevent HOT.
If you read my earlier comment properly, you'll notice a "done" column is to avoid deleting columns on the hot path and avoid dead tuples messing up the planner. I agree that a table should not contain done jobs, but then you risk running into the dead tuple problem. Both approaches are a compromise.
Nothing a well-trained model won't be able to instantly solve. It's literally just grunt work, not rocket science.
F DRMs though. Good news is those AAA games are rarely worth anyone's time anyways. Better spin up indies or classic games - a good SNES game is worth a hundred of those garbo AAA license rehashes.
I can't stress enough how much this is not true for any meaningful production threat detection software. Models regularly fail reversing even basic obfuscation tasks. Try it yourself. No, ChatGPT is not going to break Denuvo. Probably not even in OpenAI's wildest dreams.
Generating an image or sound is seemingly child's play compared to actual complex software tasks. There's not 1,000 different open source DRM codebases you can train against. It's not a diverse field.
Generating code to do script kiddie hooking? Sure. Reversing a complex multi-tiered obfuscation and trust platform? Yeah, right.
It’s hard to overstate how poor the technical quality of modern games is. Pretty much every DX12/Raytracing title is just a stuttering mess with absurd shader compilation and traversal stutter. Nevermind ridiculous ghosting artifacts and the like produced by Lumen. Modern games are optimized to look good on screenshots and not-realtime in-engine renders for trailers. Hardware and APIs have never been more powerful, and engine developers turn out the worst-running games since 30 fps hardlocked Xbox ports from the 2000s.
Can we just flag this kind of misleading and propagandist content? It’s literally content+karma farming, and brainwashing hot garbage.
Total rage bait that doesn’t have its place on platforms catering for higher educated and science oriented people.
Nuclear plants evaporates water, water rains down a few kilometers away, gets collected in sewer, recycled/filtered and re-injected in the system.
You know what doesn’t get recycled? Coal & gas burning pollution & CO2. And you know what doesn't get re-injected? Lost lives to pollution and global warming.
Rivian placed their charging port on the front left corner, while Teslas have ports on the rear left corner. So to charge a Rivian at a Tesla Supercharger, you have to park in the adjacent spot and plug in.[1]
More Supercharging stations are adding "trailer compatible" spots, so this should become less of an issue over time. I would also bet that EV manufacturers will standardize on a port location similar to how gas vehicles have. (Early gas vehicles often put the fill hole in places like the dash or under the seat.)
Longer cables have significant disadvantages. They cost more. They're less efficient, meaning they charge slower and/or require more cooling. They contain more metal, making them more appealing to thieves. They get tangled and damaged more easily. That's why Tesla made their cables so short and standardized on a charging port location.
I'm sure future charging stations will have some long cables for vehicles with odd port locations or trailers, but I don't know if every charger at a station will have a long cable. Considering the disadvantages of long cables, it makes a lot of sense for manufacturers to use one location for the charging port. That allows owners to use as many chargers as possible. And it would mean that anyone driving an EV wouldn't have to remember where the port is on their specific vehicle.
The cables are going from a cabinet many meters away from the dispenser to the dispenser. They could go another 3-4 feet and suddenly support a lot more vehicles.
Or Tesla could have placed the dispensers to the sides of even some of the spaces and alleviated the issue.
But Tesla envisioned only the few Tesla models to ever charge at Tesla chargers, so they designed them with tight tolerances.
Both of those companies have much higher costs per charger than Tesla. Kempower doesn't make their prices public, but the cheapest I could find was $50,000 for a 1 port 50kW charger. A dual port 200kW charger was almost $200,000. Alptronic's chargers are $50,000 for a 1 port 75kW charger and $90,000 for a 2 port charger with 300kW total output. Those numbers are just for the hardware. That doesn't include costs for installation, land, or grid connection. My best estimate of Tesla costs is around $41,000 per plug, and that includes all of those extra expenses.[1]
Sadly, the graveyard of failed companies is full of ones that made amazing products. Kempower's revenue has been going down over the past year, and they've burned through half of their cash reserves in the past few quarters. They've decreased headcount to reduce costs. I hope they pull out of this dive, as more competition is great for consumers. But it really is the case that fancier chargers are significantly more expensive. So much so that it's hard to make them profitable.
The chargers shouldn't have been made so short as to only ever accept 2-3 models of cars with identical charger placements. But they were designed to only work with a few Tesla models, so they weren't designed with flexibility in mind. Even Cybertrucks towing have a big issue with the vast majority of existing deployed chargers.
Lots of ICE vehicles have their gas inlets in a number of different spots. Some even have them behind the rear license plate! And yet people don't seem to have an issue getting gas in their cars.
IMO having the vast majority of chargers at the head like that was a bad decision. Putting them on the side of the spots makes a ton of sense and really opens up a lot of flexibility.
from my understanding this is addressed with the new v4 supercharger stations that are coming online now. its not just a rivian problem its the same with Ford EVs as well.
> All this stuff is just done automatically in Rust. If you have to write out the code by yourself, that's more work, and a huge chance for bugs. The compiler is doing some very complex tracking to figure out what to drop when and where. You'd have to do that all in your head correctly without its help.
What prevents anyone from dedicating a Zig memory allocator to the job (and all of its subtasks), and simply freeing the entire allocator at the end of the job? No baby-sitting needed.
Or if the mindset is really to be assisted, because “very complex” and too “much work”, may as well use a garbage collected language.
> It's knowing the compiler is on your side and taking care of all this that makes it magical.
Until you got used to it, and trusted it so much, and it suddenly misses something - either after a compiler update, or after some unsupported code introduced, and that shit takes down prod on a Friday. I’m not going to take the chance, thank you, I can call free() and valgrind.
> What prevents anyone from dedicating a Zig memory allocator to the job (and all of its subtasks), and simply freeing the entire allocator at the end of the job? No baby-sitting needed.
Given the whole ecosystem is built around the Allocator interface, it's entirely feasible for the consumer of a library to pass in a Garbage Collecting allocator, and let it do the job of deciding what gets dropped or not.
Downside is that this is all at runtime, and you don't get the compile-time memory management that Rust has.
I am sorry, are you non-sarcastically arguing that being able to pass through airport security, potentially accessing cockpits and planting bombs onboard airplanes, with a high-school level SQL injection on a federal website used by dozens of airlines & airlines employees, is actually, "fine"?
Besides, I am not sure what sort of "security through obscurity" you are talking about? Ian and Sam found it, and frankly - with a public page, page title + first h1 tag clearly stating that this relates to a Cockpit Access system, this has got to show up in a shit ton of security research search engines instantly.
We’ve maintained a financial exchange w/ margining for 8 years with it, and I guarantee you that everyone was more than relieved - customers and employees alike, once we were able to lift and shift the whole thing to Java.
The readability and scalability is abysmal as soon as you move on from a quant desk scenario (which everyone agrees, it is more than amazing at.. panda and dask frames all feel like kindergarten toys compared), the disaster recovery options are basically bound to having distributed storage - which are by the way “too slow” for any real KDB application given the whole KDB concept marries storage and compute in a single thread.. and use-cases of data historical data, such as mentioned in the article, become very quickly awful: one kdb process handles one request at once, so you end up having to deploy & maintain hundreds of RDB keeping the last hour in memory, HDBs with the actual historical data, pausing for hourly write downs of the data, mirroring trees replicating the data using IPC over TCP from the matching engine down to the RDBs/HDBs, recon jobs to verify that the data across all the hosts.. Not to mention that such a TCP-IPC distribution tree with single threaded applications means that any single replica stuck down the line (e.g. big query, or too slow to restart) will typically lead to a complete lockup - all the way to the matching engine - so then you need to start writing logic for circuit breakers to trip both the distribution & the querying (nothing out of the box). And then at some point you need to start implementing custom sharding mechanisms for both distribution & querying (nothing out of the box once again..!) across the hundreds of processes and dozens of servers (which has implications with the circuit breakers) because replicating the whole KDB dataset across dozens of servers (to scale the requests/sec you can factually serve in a reasonable timeframe) get absolutely batshit crazy expensive.
And this is the architecture as designed and recommended by the KX consultants that you end up having to hire to “scale” to service nothing but a few billions dollars in daily leveraged trades.
Everything we have is now in Java - all financial/mathematical logic ported over 1:1 with no changes in data schema (neither in house neither for customers), uses disruptors, convenient chronicle/aeron queues that we can replay anytime (recovery, certifying, troubleshooting, rollback, benchmarks, etc), and infinitely scalable and sharded s3/trino/scylladb for historical.. Performance is orders of magnitude up (despite the thousands of hours micro-optimizing the KDB stack + the millions in KX consultants - and without any Java optimizations really), incidents became essentially non-existent overnight, and the payroll + infra bills got also divided by a very meaningful factor :]
Compared to similar dynamic scripting languages, Q is very vast. Compared to statically compiled languages, it can be surprisingly competitive, but is usually slower. The truly distinctive thing about Q is its efficiency as a user interface: at a REPL you can rattle off a short sequence of characters to transform and interrogate large datasets at interactive speeds and flexibly debug complex distributed systems live. In the right hands, it's a stunningly effective rapid-application-development tool (the above "quant desk scenario"); this was perhaps even more true in the k2 days when it was possible to build ugly but blisteringly fast and utilitarian data-bound GUIs for K programs in a few lines of code. There's certainly an abundance of romanticism and mythology surrounding it, but some of the claims are real and enduringly unmatched.
And this is the architecture as designed and recommended by the KX consultants that you end up having to hire to “scale”
I think this hits on one of the major shortcomings of how FD/Kx have managed the technology going back 15+ years, IMHO.
Historically it’s the consultants that brought in a lot of income, with each one building ad-hoc solutions for their clients and solving much more complicated enterprise-scale integration and resilience challenges. FD/Kx failed to identify the massive opportunity here, which was to truly invest in R&D and develop a set of common IP, based on robust architectures, libraries and solutions around the core kdb+ product that would be vastly more valuable and appealing to more customers. This could have led to a path where open sourcing kdb+ made sense, if they had a suite of valuable, complementary functionality that they could sell. But instead, they parked their consultants for countless billable hours at their biggest paying customer’s sites and helped them build custom infra around kdb+, reinventing wheels over and over again.
They were in a unique position for decades, with a front row seat to the pain points and challenges of top financial institutions, and somehow never produced a product that came close to the value and utility of kdb+, even though clearly it was only ever going to be a part of a larger software solution.
In fairness they produced the delta suite, but its focus and feature set seemed to be constantly in flux and underwhelming, trying to bury and hide kdb+ behind frustratingly pointless UI layers. The more recent attempts with Kx.ai I’m less familiar with, but seem to be a desperate marketing attempt to latch onto the next tech wave.
They have had some very talented technical staff over the years, including many of their consultants. I just think that if the leadership had embraced the core technology and understood the opportunity to build a valuable ecosystem, with a goal towards FOSS, things could look very different. All hindsight of course :)
I'm very curious about this rewrite in Java, especially the orders of magnitude improvement. That sounds extremely impressive, and something that I wouldn't have considered possible. Can you share a bit more about how this performance improvement is achieved?
Well, I don't think the founders of that exchange complain about KDB that much.
After all KDB allowed them to go to market quickly and make billions and than they changed the tech stack when demand justified it. So what? KDB was never meant to run a large exchange, but you just demonstrated that it can run a smaller one.
> ... and without any Java optimizations really ...
Come on, be honest! All of the core tech needs to be implemented in highly optimized GC-free Java. And you need to hire senior Java consultants who are highly specialized and do that for 10+ years and they also cost millions.
I happen to know that BitMEX (located in Asia) has such consultants working from the EU. So, it's that easy to hire them!
> The punk group have all got normal jobs, which they will continue to pursue, but are pulling out all the stops for one final tour for their army of fans.
Seems in direct contradiction with this anarchist click bait title.
It's a reference to their music, although a very poorly thought out one. But I don't think the result is entirely fair, as Fat Mike is a record label owner. That's not what one may think when you say "a job", these people are all middle-aged and aren't working the cash register at a Walmart or something like that. A job is a job, but NOFX are far past some idealistic idea of "live off the music, skip college" and whatever else people thought of them.
Anarchists' portrayal of themselves is actually much more discrediting than the pop culture version; it's basically an ideology about everyone being in meetings all day.
Unlike some other ideologies they will happily answer any questions about it, it's just that the answers are extremely bad, whether it's "how would you make insulin" (you won't) or "how does criminal justice work" (sometimes meetings, sometimes lynch mobs).
Not my experience at all, esp. in terms of the people/communities I've met or the books I've read.
> Unlike some other ideologies they will happily answer any questions about it, it's just that the answers are extremely bad, whether it's "how would you make insulin" (you won't) or "how does criminal justice work" (sometimes meetings, sometimes lynch mobs).
That's precisely what I meant by pop culture/mass media. At this stage, this is more of a meme regarding anarchy than reality and this couldn't be farther from truth. (and it's quite an old one, think: French enlightenment era philosophers using pre-partition Poland as an example of anarchy).
> Unlike some other ideologies they will happily answer any questions about it, it's just that the answers are extremely bad, whether it's "how would you make insulin" (you won't) or "how does criminal justice work" (sometimes meetings, sometimes lynch mobs).
To be fair, every ideology and/or mode of societal organization sucks at handling the nasty edge cases (even the best organizational mode we've found has a saying encapsulating this fact: "hard cases make bad law"), but yeah, anarchists mainly thrive in places where they aren't subject to the consequences of bad answers (e.g. twitter, reddit, bluesky, etc).
The problem with ideologies is that they work really well in small communities.
Communism works. In a small communjty/village. Same with anarchism. Because that's basically how small villages work. You can adjust as needed in the moment without too much effort.
The problem happens when you grow. What happens when people you are governing you will never meet? What happens when you're busy and can't research all the complexities of local politics and some old person is stirring up trouble with yells for not in my backyard stuff... And one person starts bribing and so on.
The problem is that we want to find ideologies that work across millions of people. Even democracy is pretty terrible in modern countries because of how large they are, so it creates a lot of inequality and centralisation of expenditure.
I think the secret is to make the state smaller, not in the libertarian sense, but in actual geography and number of "subjects". No organisation has any business deciding the lives of tens or hundreds of millions of people. The trend is for governments to become larger and larger, just like any other empire. A one-world government, for example, would not be a utopia, it would be a veritable hell on earth.
(Disclaimer: I'm an anarchist, and as you say, anarchism works only at smaller scales, so I'm biased)
You're probably onto something. Being too big leads to problems. But also leads to power.
You can't fuck with the US/China, even if nukes were removed, because they are a massive force that will crush you beneath their heel. So there is mass power in unity. However you look at europe, and you see a lot of gains as well, where multiple smaller governments exist, and a unifying body was created to compete with the likes of the US without giving up their individuality. But also the US is distributed since states govern themselves with overarching federal oversight.
It is massively complex. At the end of the day we need not only a small anarchistic state, we also need reasons why some cult of personality won't be able to rally his million followers and start guns blazing taking over neighbors. Unfortunately I feel like democracy is the least bad system we got.
As anti-AI as I am, I think democracy might be the best we get until the day we are able to create benevolent machine kings organising our lives. Monarchies/authoritarian governments can in theory be fairer and much more efficient at tackling big problems than any democracy, but fail spectacularly, and with a lot of bloodshed, when paired with human stupidity and greed.
Until then, we're in kind of a political limbo of mediocrity.
The image I have of anarchism is not the mainstream one of "million of people doing whatever they want"; organisation and hierarchy are not in conflict with anarchism, as long as you are free to leave and form your own. So, under that point of view, anarchism-as-political-force is little more than a multitude of small heterogeneous communities collaborating and trading with each other. Maybe the secret is to embrace our tribal nature, but avoiding the issue of cult of personality—this is an interesting dilemma to which I don't have a good answer for, so thank you for the food for thought.
The problem of "free to leave" is that that only works with unlimited resources. Why should I allow a random person into my house to eat my food and promise to assist, only to be free to leave at any time. And they have to go to another house, why would someone there do the same.
Being altruistic is great, but at some point you run out of necessary resources. And altruism dies at scale when resources dry up.
> "Controversies aside — most of which involved drugs, onstage banter taken too far and the unpredictability of both the band and its fans — the members of NOFX managed to do something most people can only dream of: They avoided having a day job for 40 years."
We're going to need some deep investigative journalism to sort this mystery out...
Also, I remember reading when I was young that NOFX quickly abandonned the "drugs/alcohol/sex" ethos and become quite healthy people despite having the opposite branding.
It hasn't exactly been a clean path. Fat Mike has been on and off through the years and they never stopped singing about doing drugs. That said I think they are vegetarians and otherwise relatively health conscious.
Milo Aukerman of the Descendents has a PhD in molecular biology - there's a great interview (maybe in the Filmage documentary?!) where he's discussing a conversation with his colleagues who are asking "what did you do at the weekend?" and he replies "Oh, my band was playing Woodstock..." (or something along those lines!)
Interesting, did not know that. Punks seem to like biology it seems ..
"Graffin obtained his PhD in zoology at Cornell University and has lectured courses in natural sciences at both the University of California, Los Angeles and at Cornell University."
They've toured for 40 years, sold 8 million records and have over a million monthly listeners on Spotify. The singer's label has put out many of the big punk bands for the last 30 years. I agree with your point, but this lot are fine.
They are indeed “doing just fine” which I am sure is what you must have been referencing — from one of their most famous songs:
Buy me a Becks beer
or pass me the bong
Gimme some Bushmills
I'll sing you this song
Open another
big box of cheap wine
We're over 30
we’re doing just fine
The last time I saw them live it was pushing 40 instead of over 30 and that was nearly a decade ago.
Hahaha, yeah, they're a fun band. I saw them (for the last time, I guess?!) a couple of weeks ago (and oddly in reference to the lyrics, I actually saw them tour Pump Up The Valuum as well). Mike is pushing 60 now!
It's not entirely different from all the successful artists who are living lavish lifestyles (for example many musicians are complete car nuts and own insane cars collection [1]) but sing songs criticizing "money" (even though they have plenty) or criticizing "wall street" (even though most of their savings, in addition to cars and real estate, are at their broker).
I have nothing against money or wealth.
But the irony of a musician or a band criticizing money while flying private is not lost on me.
[1] as an example of such musicians (but not that they necessarily did criticize money), Miles Davis used to run vs Herbie Hancock in the streets in a "Ferrari vs AC Cobra" style. And I love that.
To be fair, I don't think anarchism (at least as espoused by the CNT-FAI et al) suggests that anyone can just sit on the couch without doing anything.
If you want that, may I recommend capitalism?
If your expenses are in USD, 30Y TIPS are above 2% last I saw, so putting 50x your burn rate in those should allow you to sit right back, clip those risk-free coupons, and watch the world go by.
I don’t get it. If you do not want to participate in the preservation and distribution of the archive, why don’t you just move on instead of complaining?
Besides, gluetun+chihaya+qbit containers do the job without breaking a sweat, and without ever having to remember that you run a VPN - as it’d only be tunneling the containers of your choice. gluetun is the best image ever made!
Hang on. I think the OP is trying to find a middle ground between their level of comfort and contributing to the project. I think their intentions are positive; they want to help but are constrained by their other needs or priorities.
Everyone’s talking about how you MUST hit the database for revocations / invalidations, and how it may defeat the purpose.
How is no one thinking of a mere pub-sub topic? Set the TTL on the topic to whatever your max JWT TTL is, make your applications subscribe to the beginning of the topic upon startup, problem solved.
You need to load up the certificates from configuration to verify the signatures anyways, it doesn’t cost any more to load up a Kafka consumer writing to a tiny map.
For maximum scalability you'd want a bloom filter at each service for testing the token, and some central revocation lists where you go test the token that fail this.
But this is way overkill for anybody that isn't FAANG, and it's probably overkill for most of FAANG too. On normal usage, it's standard to keep the revocation filter centralized at the same place than handles renewals and the first authentication. This is already overkill for most people, but it's what comes pre-packaged.
If you have a pub sub cluster that you push revocation details into, and running servers subscribe to that feed and then track a rolling list of tokens that would otherwise still be active but have been revoked, you are effectively storing a revocation database on every edge node.
Again not really. If the pub/sub broker is persistent, you don't have to persist the revocation list on edge nodes. And just pointing out that it's a db in some loose sense of the word doesn't help with the actual challenge of organizing the flow of data reliably in a federated system (i.e. one that can't share a single database).
As of PG16, HOT updates are tolerated against summarizing indexes, such as BRIN.
https://www.postgresql.org/docs/16/storage-hot.html
Besides, you probably don't want "done" jobs in the same table as pending or retriable jobs - as you scale up, you likely want to archive them as it provides various operational advantages, at no cost.