Fake tracking numbers are a common occurrence in scams now. Somehow the scammers are getting access to a database of real time legitimate tracking numbers, they wait until there’s a shipment in their database going to the same city as the buyer, and then use that tracking number to claim that they shipped the package. Maybe they’re paying a real merchant for access to their shipping database? Or are UPS tracking numbers short enough to brute force?
There are no till drills that are being sold exceptionally cheap (~$800 US) and show up as the first google ad that are rumored to be an empty box shipped from the country of origin (not China) with incorrect paperwork that gets stuck in customs. Most people say they get their money back through CC or Paypal but maybe a few don't check? Either way it was plausible enough that I did not purchase. The market is small farmers and hunters planting food plots. You can use an old repurposed grain drill if the grass is super short/overgrazed and you can fix it, ~$1500 used on marketplace, or buy a small 4' (no tires/3 point hitch) no till drill US made low end ~$8k. So it is "too good to be true".
Higher frequency means smaller and therefore lighter transformers, which is very important on aircraft. Nowadays a DC voltage supply would be better, but DC to DC voltage converters didn't exist when the 24V 400Hz standard was created.
For mains voltage we use 50-60 Hz because lower frequencies work better with very large AC generators in power plants and, and lower frequencies are more efficient to transmit long distances.
Siemens NX used to be available on macOS, up until about 5-6 years ago. However, Siemens found few customers were adopting it, so they discontinued it.
If Apple really cared about it, I'm sure they could have worked out some deal with Siemens to keep the macOS edition viable, even improve its UI to be native and more usable. It would require them to move away from viewing themselves just as a customer, into more seeing it as an investment into their own platform (making it viable for more use cases.) Sad to say, Apple doesn't seem to think like that much (or maybe some people there do, but not the people whose opinions really count)
Why Is it that the Mac Pro has so little Pro software available for it? Could it be the neglect? AV software used to be Apple's tentpole for Pro's, until they decided Prosumers were more profitable and lost mindshare of actual Pro's as a result (see the great migration after the releasee of the first Final Cut Pro X)
For CAD specifically I'd say "high end" are CATIA, NX and maybe Creo? SolidWorks and SolidEdge would form the tier below that.
For CAx in general it's difficult to say, because there's such a huge variety of software and every little niche has its own highly specialized (and usually very expensive) tools. I've never seen any of these support anything other than Windows and/or Linux. Some of them have Win32 GUIs and Linux is headless only (for running simulations), some are fully cross platform, some are Linux-only.
Most CAE engineers around here work on Linux, mainly because they prefer how easy it is to script and automate things and because the headless HPC/batch environment is of course also Linux. Meanwhile CAD and design only happens on Windows because CATIA is Windows only, like the various Autodesk tools used for CAID. From the late 80s to the early 2000s this place had a ton of UNIX workstations (SGI, HP, Sun) and even UNIX clusters (IRIX and SUPER-UX among others, the latter having virtually no representation on the internet today). There's also still IBM AIX systems around, as well as IBM mainframes. Not my department though.
> There's also still IBM AIX systems around, as well as IBM mainframes. Not my department though.
Are you saying some people still use IBM mainframes for CAD/CAE/etc applications? If yes, that's unexpected yet intriguing information, and I'd love to know more.
Although maybe you were just stating the obvious that IBM mainframes survive in general even if no longer in this particular domain.
Thanks for your answer! What makes it so that AutoCAD is not high end? Like what features does it lack or what workflows does it not support?
Suppose something like CATIA was available on Mac and offered lets say a performance benefit. Would you consider Mac? Or would you for example still need a whole other set of tools to be available as well for it to be even possible?
I have seen some posts on the Intel FPGA community forum about using Quartus Prime Pro (FPGA design software) on Apple Silicon. Apparently, it works, even though not officially supported.
I haven't had the time or the desire to take it for a spin yet, but I'd love to be able to compile FPGA designs on a nicely configured and quiet M2 system.
So either that's a good sum to their standard of living, or the data isn't all that personal and private in the first place.
The transit nerd in me kind of wants to buy it, not for maliciously tracking the customers, but to analyze the commuter and intercity travel patterns of 30m people.
IRCTC does not generally have much pii, except for adressess, phones, email. Aadhar can be linked but that on its own is not useful. Moreover a lot of accounts are burner accounts so the data is likely to be fake.
Does anyone buying an "exclusive" copy actually believe they're getting that?
The pricing is also interesting from another angle, if we take them at their word ("5 copies" vs "exclusive"): why accept 25% less overall? Assuming this is a cryptocurrency transaction, it surprises me the simplicity of dealing with a single buyer is worth that. Can anyone with more insight explain?
The way they went from GPT-3 to ChatGPT is really quite genius. My understanding is that it's something like this:
1. Start with GPT-3, which predicts the next word in some text and is trained on all the text on the internet
2. Take thousands of prompts, generate several responses for each of them, and have human reviewers rank the responses for each prompt from best to worst
3. The GPT model needs a massive amount of training data, it would be cost prohibitive to get enough human feedback to fine tune GPT manually. So you train another model, called the reward model, to predict how the humans will rate each response. Then you train the GPT model against the reward model millions of times
5. Feed a small percentage of the output from that training process back to the human reviewers to continue training the reward model, based on heuristics like reward model uncertainty which predict how helpful the human feedback will be towards improving the reward model
6. Release ChatGPT to the public, and use user feedback like response upvotes/downvotes to further optimize the reward model, while continuing to train ChatGPT against the reward model
> 2. Take thousands of prompts, generate several responses for each of them, and have human reviewers rank the responses for each prompt from best to worst
Step 2 is not that. It's manually writing responses for a few tasks.
> A labeller demonstrates the desired output behavior.
So it is supervised training in this stage. Ranking is the next stage, for training the reward model. This is not the reward model, it's a model to generate sample responses to be used by the reward model.
So there are two kinds of manual work involved here - manually demonstrating how to solve tasks, and ranking responses. There is even talk about how much effort to invest in the first vs the second and what is the trade-off.
Right I intentionally left off Step 1 from that chart to simplify the explanation, since it didn't seem necessary. Is Step 1 just for creating the ChatGPT content blocker?
I want to know if it will ever be possible to run this kind of AI at home once its training is complete. I dont need all the knowledge just subset that I'm interested in.
Actually I'm more interested in its ability to transform things. For example I can ask it to convert docker-compose to docker run command, it can manipulate JSON, it can sort numbers in table when prompted. I'm more interested in these abilities rather than just getting answers for which I already have Google
This is honestly affordable for a lot of upper-middle class people and might well it worth it. It's like the cost of a car. I can seriously see this writing a book for me if I can get it tuned to study only my writing style and remember all of my texts. But it could also only cost $14000 14x RTX 3090s.
Difference is in first mover advantage. If you can be the first to use it to bring value to yourself and your clients, you can easily make up the cost of that hardware.
If this was open sourced it may be quickly optimized, the amount of VRAM required for image generation went down very quickly, I'm sure Dalle-2 is still using enormous vrams but other solutions are not.
I think he wants to self host. It sucks to have no ownership of such a powerful tool I would pay upwards of $3000 to be able to self host something like this.
Rest assured someone is working on a self-hosted (distilled) model. Stable Diffusion has shown there is a viable market for open, consumer-hardware inferencable models.
ChatGPT seems to be/result in some amount of caching of responses - there is very little variation when to asking the same question multiple times. CharacterAI produces a lot more variety in comparison, making it more helpful for brainstorming. That said ChatGPT is likely closer to the truth, even if not perfect, for searches. The innovation happening lately is incredible.
There's definitely some live pruning happening, but another factor is that the temperature is turned way down. Obviously at a low temp it's just a totally deterministic function, and if it's doing it's job you'd hope that similar questions would be mapped very close together in the configuration space
> Take thousands of prompts, generate several responses for each of them, and have human reviewers rank the responses for each prompt from best to worst
Recently I saw an image where Indian women sat in front of computers and the caption said they were classifying "AI" responses. I guess that's true and this kind of work is the new outsourced cheap labour in the AI age.
That Indian woman's idea of acceptable and not acceptable AI responses surely vary from that of a San Fransisco tech worker, or Cape Town motorcycle mechanic, or an English teacher from Liverpool.
I really doubt the mechanical turk method is applicable or even useful for the current state of AI-generated text.
i actually disagree a lot with this. Sure, if you asked something with heavy cultural baggage that would frequently be a real concern, but when you are primarily trying to bridge the machine-human chasm, our cultural differences among the examples you gave are trivial in comparison. For instance, if you offered an AI personal assistant but the catch was that it would (at least starting out) only have the perspective of an average middle-class Indian person, it would still beat the absolute crap out of "first generation" technology like Siri or Alexa!
That's super interesting. When GPT-3 came out, I wrote an article inspired by it. That we could one day build an AI that acts like AGI, by a crazy vast amount of multimedia training data, collected by willing users to participate in ever improving AI interactions;
When I was a first year AI student beginning of the 90s I asked my professor what would happen if we just made a massive neural network and trained it with all information in the world. He said it cannot happen as it it impossible.
This reminds me of a major US newspaper declaring heavier-than-air flying machines a million years away mere months before the Wright brothers experiments.
If an elderly but distinguished scientist says that something is possible, he is almost certainly right; but if he says that it is impossible, he is very probably wrong.
Considering the computational resources available at the time, he was not that wrong. Research into artificial neuronal networks has always been held back by available computational power.
Agreed, but as a professor I believe one needs to be looking in the future. It was not that far out, but yeah it was an AI winter. We were stuck until 2012 basically. That's a long time.
I'm a professor in an AI field, and I can tell you that neither myself nor the colleagues I regularly have scientific discussions with could imagine ten years ago that something like ChatGPT would be possible in 2022. I suppose there might be a minority who called it, but recent advances in deep learning absolutely whooshed past the predictions of the overwhelming majority of people in the field.
Ah yes, that's what I was trying to say. I was the worst sceptic of AI; I never did anything with AI with it after getting my masters. I just went for money, programming and managing programming.
For me [1] this is the most mindboggling thing I have seen in my life and I don't think people realise what it means. And yes, it wooshed passed anything I thought possible in my lifetime. I hate that it's 'not be evil', 'anti thought crime' etc but it is really incredible what it does.
I think the issue here was with the term genius, which makes it sounds like what was a completely new paradigme and revolutionary.
OpenAIs success mainly stems from extremely well executed previous concepts while mostly ignoring cost. And as they're pretty much the most successful public player in this domain, they've got the first-mover advantage which they're currently very succesfully leveraging. At least thats how it looks from the perspecitve of an armchair analysts, which wouldn't have been able to achieve the same -- even if I had the same resources and time.
The actual result is absolutely incredible however, regardless wherever the road to this end was genius or not
I don't think anything about high-performance GPT models is standard, since they are only a couple years old and only a handful of organizations have developed them
The technique in question has little to do with GPT itself; it involves using ML to generate more training data in an automated fashion, creating a generative training loop, which as another commenter mentioned, is also the basis behind general adversarial networks.
Yes, some variation of this iterative approach has been around for over 2 decades. Look at Oren Etzioni's early work on Open Information Extraction and so on.
I agree that applying this semi supervised approach to extend GPT for improving dialog models is what is unique & results are stunning. But the meta method has been around for a while.
In learning to predict the next token, the model has to pick up lots of little bits of world knowledge. I'm sure someone would disagree with the phrasing of "understand", but it certainly operates with more complexity than, say, a markov chain. It has seen lots of python, and in order to predict better, it has developed internal models of how python works. Think of how much better you'd do predicting the next character of python code compared to random noise- there's a lot of structure there.
In my (limited) experience it seems to perform even better for typed languages (for example Kotlin/Java/Swift) compared to Python. The Python code it provided often had subtle type issues when working with dates. While the Kotlin date-related code it provided was more accurate and correct in terms of types. Which makes sense since the additional type information likely leads to a much better "internal model of how Kotlin works"
What surprised me was the level of "understanding" it seems to do when providing it with some of my own sample code. It can analyze the code, explain how it works/what it does, use libraries, suggest improvements and apply those improvements.
While the end result isn't perfect, it's still highly impressive and while I was an AI-skeptic before, I now see the possible benefits of AI assistants for programming.
Some other prompts with very impressive results:
* "Write an implementation for the following Kotlin repository interface: <insert-interface-with-full-type-signatures>."
* (followup) "Add save/load methods that store the backing map in a JSON file"
* (followup) "Replace Gson with Jackson for JSON serialization"
* "Write an Android layout xml for a login form with username/password/loginbutton"
* (followup) "Provide the Kotlin activity code for this layout"
* "Write a Kotlin function that parses a semver input string into a data class"
In my (limited) experience it seems to perform even better for typed languages (for example Kotlin/Java/Swift) compared to Python. The Python code it provided often had subtle type issues when working with dates. While the Kotlin date-related code it provided was more accurate and correct in terms of types. Which makes sense since the additional type information likely leads to a much better "internal model of how Kotlin works"
I think another possibility here is that they might have used an execution environment to check whether the code the model came up with actually compiles and used that as additional input during training. Some sort of execution environment seems to me to also be a possible explanation for how they managed the model to emulate a terminal so well.
It’s not ‘more complexity’ than a Markov chain - it essentially is a Markov chain, just looking at a really deep sequence of preceding tokens to decide the probabilities for what comes next.
And it’s not just looking that up in a state machine, it’s ‘calculating’ it based on weights.
But in terms of ‘take sequence of input tokens; use them to decide probable next token’, it’s functionally indistinguishable from a Markov chain.
I look at deep sequences of tokens and predict what comes next- can you milk me? Once you've broadened "basically a markov chain" to "any function from a sequence of tokens to a probability distribution of tokens" there's a lot of explanatory power lost. If you had to characterize the difference between brute force mappings based on pure frequencies and model which selectively calculates probabilities based on underlying structure, wouldn't you say the latter had more complexity?
You don't have to believe the hype, but if you think you can get GPT performance out of anything remotely resembling a markov chain, I encourage you to try.
There's nothing about Markov chains that says the model has to be based on brute calculation from previously observed frequencies. The point is that the exact behavior of these LLMs could also be modeled as a Markov chain with a sufficiently massive state machine.
Obviously that's impractical and not how LLMs actually work - they derive the transition probabilities for a state from the input, rather than having it pre-baked - but I think from the point of view of saying 'these are more sophisticated than a Markov chain', actually strictly speaking they aren't - they are in fact a lossy compression of a Markov model.
But it seems like the attention mechanism fundamentally isn't markov-like in that at a given position it can pool information from all other positions. So as in the simplest case when trained on masked-language modeling, the prediction of the mask in "Capital of [MASK] is Paris" can depend bidirectionally on all surrounding context. While I guess it's true that in the case where the mask is at the end (for next-token completion), you could consider this as a markov model with each state being the max attention window (2048 tokens I think?), but that's like saying all real-world computers are FSMs: it's technically true, but this isn't the best model to use for actually understanding its behavior.
Since for most inputs that are smaller than the max token length you never actually end up using the markov-ness, calling it a markov model seems like it's just in a way saying it's a function that provides a probability distribution for the next token given the previous tokens. Which just pushes the question back onto how that function is defined.
Could you not use two Markov chains for masked language modeling? One working from the beginning until [MASK] and one working backwards from the end until [MASK]. And then set [MASK] to the average of both chains. If a direct average cannot be found, it is assumed to be a multi-word-expression and words are generated from the two chains until they match.
It's really awesome how good it is in modeling certain world knowledge. It seems to be struggling with putting everything in one framework. For example, it still makes a lot of mathematics and logic errors.
Make a large enough model and train it with all sorts of data and it will be able to encode generalized concepts which can then be applied to specific tasks (given only a few examples of the task, or even just a query / question, rather than an example)
> 6. Release ChatGPT to the public, and use user feedback like response upvotes/downvotes to further optimize the reward model, while continuing to train ChatGPT against the reward model
Can someone provide a pointer to an article that elaborate this part?
How does step 1 work? It seems incredibly inefficient to check your word combo against every single segment of text they have. How does it do this efficiently?
A friend's 18yo brother was motorcycling in the mountains with their father, crashed and broke his femur. Ambulance would have taken hours, they had helicopter rescue insurance, but the only helicopter company that operated there wouldn't take it. Got a $25k bill for the helicopter ride and negotiated down to $16k iirc.
One theory is that the crypto quant funds figured out how to exploit the Alameda FTX market maker starting in 2020-2021 to take tons of money from Alameda. But FTX couldn't just turn off the Alameda market maker because most of the FTX trading volume, and therefore FTX revenue, was these crypto quant funds taking money from Alameda. So if they turned off the Alameda money spigot then their revenue would drop off a cliff and they wouldn't be able to raise more money from Sequoia or the UAE. And the value of FTT was tied to the trading volume and was a huge portion of their assets, so if volume fell they would be insolvent.
Basically they turned customer deposits into revenue at pennies on the dollar.
This doesn't make much sense to me. I think the problem is that arbitrage just dried up (as it does in a maturing market), and SBF got high on his own farts and started making directional bets. This is clear from the amount of FTT tokens that were held by Alameda in comparison to the amount of assets a typical market maker would hold relative to a pair's liquidity.
This is what likely happened!! These group of inexperienced guys thought they had found a better way to market make, they even had a blog explaining their better strategy.
But 5 minutes of reading it immediately becomes obvious that it can easily be exploited and leave alameda holding the bags.
You can't be the exchange, the person loaning out huge margins, and the person backing up the entire system, all in one. Too many conflicts of interests, that leads to making suboptimal decisions. One part of you has to keep the process propped up, so that the value of margin loans you gave out don't fall, which means eating up a lot of lot of losses from the people you gave margin loans too. And you can't stop eating up the losses, or the whole thing collapses.
This was always going to fail. The only play they had was to keep the crypto hype going so that something other than them will drive the prices up and they can exit all the losses they took on from market making quietly. And that was the play they were going for. Hence all the SBP hype, the superbowl ads, the naming of sports arena. Excess crypto hype was their only play. And that's all they did the last few months. Find ways to get publicity for crypto, SBF etc
Yep, SBF was 6th on the list of largest 2022 donors by donating exclusively to Democrats. Guess who's 14th on that same list donating exclusively to Republicans:
High impedance (low current flow) signals are more prone to electrical interference. This is particularly bad when you're running long wires across a factory with big electric machines running. And if your signals are low impedance (significant current flow), then you get voltage drop across your long wire, so you can't use voltage-based signaling.
