I've been a beta tester for a few weeks now. And as a Search and Discovery person I have plenty of granular feedback and feature requests for Dan, Brett and team that I still owe :)
Overall it has helped me in a couple situations. I will say when I was onboarded I don't believe audio was mentioned and a quick search for things I know I said in Zoom recently comes up empty--perhaps that functionality is new?
For now, I've found that I haven't yet hit the equilibrium point where I've forgotten something badly enough to replace my instinct to use Spotlight, browse Finder, or even search Google Drive (shudders). The Fitness app on iOS needs 180 days of data before it shows you trends; I feel like Rewind is similar in that I need a critical mass of data and time before I truly don't know where to look and go to Rewind first. Audio support would greatly reduce that window, however, so maybe I am missing out.
Overall I definitely recommend trying it out, however!
Congratulations on the launch! Looks fantastic! I have built a product https://gettrici.com that records your screen and stores the recordings locally as well. As a developer, watching a quick replay of my recording helps me recover my lost train of thought very very fast and saves me a ton of time. [ Show HN Post: https://news.ycombinator.com/item?id=29277671 ]
Its great that you have leveraged the power of Apple Silicon to analzye and index the recordings as well as keeping the size of the recordings very very small. Do you see this becoming available for Intel Chipsets a.k.a Windows and Linux anytime soon?
Claim: For your privacy, we store all of the recordings locally on your Mac. Only you have access to them.
From the docs:
> Video data is never sent off your Mac. The only data sent to a cloud service is the audio. We send it to a cloud transcription service in order to generate a transcript of what was said. This transcript is created so you can read it and search for specific words that were said. [1]
Yeah, we only "store locally", but hey, that doesn't stop us from sending it to a third party service for transcription. I was already uncomfortable with the idea of being voice recorded. Reading between the lines of how it works doesn't inspire a ton of confidence.
The reason we wrote that article is to be 100% transparent about what happens to your data. No reading between the lines necessary.
The only time this cloud-based transcription is used is during a Zoom meeting, which is already in the cloud.
When you join a new Zoom Meeting we prompt you to ask if you want to record & transcribe the audio. You can choose to do this selectively for each meeting, or not record & transcribe any of them if you aren't comfortable with cloud-based transcription.
That said, we do plan to move away from cloud-based transcription, likely with Whisper from OpenAI.
It would help to tweak the marketing a bit around recording "everything you say." Is the mic always on? Is it always recording in the background? Is it just for Zoom/Video Conf meetings? Because the first impression I get is that of a scary Black Mirror episode.
Yea my first impression when reading the landing page was also that this is exactly a Black Mirror episode: "The Entire History of You": https://www.imdb.com/title/tt2089050/
You are right, it's a bit too aspirational right now. The "everything you say" part is just for Zoom meetings right now. Once we implement local transcription with OpenAI Whisper then I think it makes sense to expand beyond meetings to things like YouTube videos, Podcasts, Audiobooks, or anything else you might want to search by "what you've heard".
The "memex" in various form is a classic idea many tried to implement with the best tech of the moment, BUT there is a big BUT: I can't trust modern hw, full of closed source gazillion of SLoC firmwares, with networking capabilities and normally connected everywhere. Similarly I can't trust a third party closed source OS or EVEN a FLOSS one not developed by a vast community but by few big entities centered in a country (no matter witch, mine included).
In the modern world even the most genuine and honest people can't be trusted on black boxes so such kind of memex while wonderful and potentially useful have a so big safety trade off that's a no-go for me... Personally I do not even live my smartphone near me at home since I consider it a macro-spy tool operated by someone else. I can't do much for in-car stuff since I need cars and can't live on historic one for mere comfort. But more than that is beyond personal Overton window of acceptability...
Anyway, that's a very nice implementation of a very nice idea.
you got downvoted for it but it's a valid point. if you have an objection due to the privacy aspects then you're not in the target market. TAM is big and doesn't include you at this juncture.
Feels great to see ambitious products like these pop up every now and then. I feel like there are too many boring startups nowadays. Yes, there's a big brother feel to it, but I believe it's a feature, not a bug. Otherwise it might not be innovative enough...
Edit : do you hire EU citizens ? :)
Edit edit : most of the world doesn't use macOS : what's your long-term vision for non Apple silicon users ? On smartphones ? Any plan on a hardware product that literally records everything you say ?
I really admire the ambition of this product. I've noticed how much I use the internet as a "second brain". Being able to quickly get back to things would improve my life.
The UI also looks very good for a first version. Excited to give this a try!
Looks like this idea was inspired by a Show HN that appeared last year [1]. Seems the original inventor of the concept didn't even realize what it could become, or didn't have the funding to make it happen.
This looks awesome. The Entire History of You is my favorite Black Mirror episode and yet I still want this personal search functionality and much more.
Interesting product. I will take a wait and see approach though because my trust of companies pinky promising not to upload my data, especially when dealing with such invasive data as camera and audio and screen recordings, has been wearing thin of late.
I've been recording my desktop on Mac, Windows and Linux, recording audio in my office, including transcription of the audio, and multiple camera capture for over a decade since early 2000's, and using a SenseCam-like device, on-and-off, for almost 20 years. My current setup of an NVidia Jetson does an admirable job of determining if one or more people are in the room, the activity they are engaged in, multiple camera angles, transcribing what is being said, music identification, desktop image capture and OCR of the images is very much kept locally and I can be assured nothing is leaking out without my explicit knowledge. I have a custom device on my desk that I built that can "pause the recording" or "forget the last 30 seconds" or "forget the last five minutes" for those events that probably shouldn't be captured, e.g. I'm picking my nose and carefully inspecting whatever was retrieved.
Wow - would love to hear more about the hardware/software stack you're using to perform all this. And how do you go about it when away from the office?
I've started some work on a similar project as well on Windows. We should be able to fully query our lives, and we should also be guaranteed that data is ours. I believe this is where data collection should go - your data is yours, and you own it. All the data that Google, Facebook, Apple, Amazon, etc collect on us, yet we have such limited access to it. This should be ours, and ours to analyse, search, and gain insights on in any way we wish. This should really be an open-source, community project as well - tools, analytics, apps, all made to help you be a better version of yourself.
I'm pretty sure this is a BS marketing claim. Their site mentions "We compress raw recording data up to 3,750x times". This is just the regular hardware supported h.265 codec that comes in every mac. They're claiming "up to" because there are compressed frames such as B-frames that are the smallest frames which can typically be as small as 2000 - 4000 bytes. So if you look at an uncompressed 4k frame (3840 pixels * 2160 pixels * 10 colordepth)/ 8 = 10.368 MB. So an uncompressed frame of 10.368 MB / 2700 Bytes (H265 B-Frame) will give you ~3750X compression.
I could be wrong but I'm skeptical about this claim because video compression can't be done so simply just by creating a new algorithm and writing it in software. Especially the newer codecs such as h265, h266, and AV1 must be in hardware or else it would be too computational heavy and your device will just choke up trying to spend so much cpu time compressing when you're trying to run other processes.
Also if it was their own video compression, they would have to write their own decoder for their own player for every device. That is way too much for a start up to take on. It just smells like BS to me.
> I could be wrong but I'm skeptical about this claim because video compression can't be done so simply just by creating a new algorithm and writing it in software. Especially the newer codecs such as h265, h266, and AV1 must be in hardware or else it would be too computational heavy and your device will just choke up trying to spend so much cpu time compressing when you're trying to run other processes.
This is the key information, as they support M1/M2 which have en/decoders in silicon, they definitely use some off the shelf codec. So they don't use any new breakthrough tech, but marketing existing tech in a shady way.
Totally understand how it might feel that way initially. Rewind is built privacy first. Unlike other cloud-based products, all of the recording data is stored locally on your Mac. It's not accessible to others (or to us as employees).
Not only that, but we have the following features focused on privacy:
1. Full control: users can pause and delete recordings at any time
2. Exclude apps: users can exclude specific apps they don’t want recorded (e.g. Signal or 1Password)
3. Incognito: by default, we don't record Chrome Incognito or Safari private browsing windows
We have a long roadmap of things we have planned to further our commitment to privacy. Anything in particular you'd love to see us implement that might make you feel more comfortable?
These are valuable and important, and perhaps the best that can be done, but this also reminds me of when Google Glass was first announced. People did not like the idea that the person you're talking to could be surreptitiously recording or photographing you. Similarly, people hate that Facebook may have a face recognition fingerprint for your face even if you don't use Facebook.
Recording meetings for later searching still feels weird in a similar sense. Even if I trust the way my data is handled when I use it, it's a violation of privacy norms in a way that feel "creepy" to be making a persistent record of things other people said to me in what is traditionally an ephemeral context.
(That said, this sounds like something I've occasionally wanted for a long time.)
Meetings might be touchy but just recording the web browser and having the ability to search the full text would be wonderful.
I'd like two AI toys on top of it - embedding similarity search and question answering. The AI part can be executed easily with HuggingFace. There is a great QA model (FLAN) trained with multiple tasks, it can sort of do what GPT-3 does but locally, the private & cheap option.
That could be said of anything one does with a computer. The point is that if it's stored locally YOU have responsiblity over it, rather than trusting to some other party to hopefully protect it (and not sell it or expose it or what-have-you).
Side note the landing page with it's hard gradients and shadows looks like something from the late 2000s when you would buy Mac apps without the App Store
That was me. The site definitely has some areas I need to refine, but I'm bringing some gradients and shadows back. We've lived in a flat world for far too long.
It’s a macOS app that enables you to find anything you’ve seen, said, or heard.
We’re also announcing that we’ve raised $10m at a $75m valuation led by Andreessen Horowitz.
In case you are curious, the inspiration for Rewind came from an unlikely place:
Our founding story
I started to go deaf in my 20s. When I turned 30, a hearing aid changed my life. To lose a sense and gain it back again felt like gaining a superpower. Ever since that moment, I’ve been on a hunt for ways technology can augment human capabilities and give us superpowers.
That hunt ultimately led me to memory. Studies show that 90% of memories are forgotten after a week. Just like our hearing, our memory gets worse as we get older. But does it have to? If we have hearing aids for hearing and glasses for vision, what’s the equivalent for memory?
What if we could use technology to augment our memory the same way a hearing aid can augment our hearing? This question is why we founded Rewind. Our vision is to give humans perfect memory.
Introducing Rewind
Here’s how it works:
1. Local and Private By Design: We record anything you’ve seen, said, or heard and make it searchable. For your privacy, we store all of the recordings locally on your Mac. Only you have access to them. Learn more: <https://help.rewind.ai/en/collections/3616305-privacy>
2. Mind-Boggling Compression: Storing all the recordings locally means compression is very important. We compress raw recording data up to 3,750x times without major loss of quality. For example, 10.5GB of raw recording data becomes 2.8MB. That means that even with the smallest hard drive you can buy from Apple today, you can store years of recordings.
3. No Cloud Integration or IT Required: In order to enable you to search for anything you’ve seen, we use native macOS APIs and Optical Character Recognition (OCR) to recognize & index all the words that appear on your screen. This means you don’t need to integrate with cloud products like Gmail, Dropbox, or Slack. Rewind starts capturing these apps right away, with no IT required. Also, OCR enables you to copy & paste anything from your past.
4. Meetings: In order to enable you to search for anything you’ve said or heard, we use state-of-the-art Automated Speech Recognition (ASR). This is particularly useful for meetings, where the content of discussions, debates, and decisions are often lost forever as soon as a meeting is over. With Rewind, you never have to worry about losing this content again. Rewind can capture the content of meetings, including everything that is said and everything that is shared visually. All of the content is searchable. You can go back to the exact moment in a meeting you are looking for by simply searching for a word that was said or a word that appeared on your screen. Learn more: <https://help.rewind.ai/en/articles/6526620-recording-meeting>
5. Unleashes Apple Silicon: The key enabling technology that makes Rewind possible is Apple Silicon. We utilize virtually every part of the System on a Chip (SoC) so that running Rewind doesn’t tax system resources (like CPU and memory) while it is recording. It feels virtually imperceptible.
Rewind is just the first step toward our vision of giving humans perfect memory.
Please share any and all feedback you might have! We’re all ears
Before you record someone you should always ask for their consent. This is especially important with Rewind, since the recording happens on your Mac and other meeting participants won't know they are being recorded unless you inform them.
How should I ask for consent?
Here is how we recommend you ask for consent:
"This meeting is important to me so I want to make sure I don't forget what we discussed. Is it ok with you if I use Rewind to record and transcribe the meeting? For our privacy, all data is stored securely on my Mac and not in the cloud."
In addition to saying this verbally, we recommend including this message in your calendar invites so people have a chance to think about it ahead of time. Feel free to link to this help center article as well.
What are the laws around consent?
The laws pertaining to recording other people depend on where the people being recorded live.
While the laws differ from state to state, we believe privacy is so important that we recommend users of our product hold themselves accountable to a much higher standard than the bare legal minimum. We believe all users of our product should proactively seek consent from everyone they record even if they are not legally obligated to do so. Not only is this is the safest approach legally, but it is also just the right thing to do.
If you are more comfortable removing the word Rewind and changing:
"This meeting is important to me so I want to make sure I don't forget what we discussed. Is it ok with you if I use Rewind to record and transcribe the meeting? For our privacy, all data is stored securely on my Mac and not in the cloud."
to
"This meeting is important to me so I want to make sure I don't forget what we discussed. Is it ok with you if I record and transcribe the meeting? For our privacy, all data is stored securely on my Mac and not in the cloud."
Dan! Can't find the tweet (bc I didn't have rewind yet), but Patrick Collison described this exact product in a tweet once. Amazing work. There's a mic drop moment in the demo when you're able to click on the link shared in the zoom, totally sold me.
If you could bump me on the invite list (mike@luabase.com), would truly appreciate it. I'm a good beta user, promise.
Personally/imho/since you asked/nitpicks - point 2 and 5 are overly technical. For 2, it's a really great thing that the data compresses down, but my local machine has enough space that I don't think about local storage because I've not run out of space. (yet, on my M1 macbook air.) I'd lead with "you can store years of recordings on the smallest hard drive apple sells". Maybe mention how awesome your compression is as enabling technology.
For 5, start with "It feels virtually imperceptible." rather than end with that, and don't define/use SoC. Maybe something like "Unleashes Apple Silicon: Thanks to the advancements with Apple Silicon, the overhead of Rewind is virtually imperceptible by utilizing every part of the chip including the Media Engine."
Really intrigued by the product and hope I can try it soon!
Have you ever read Ted Chiang's 'The Truth of Fact, the Truth of Feeling'? What if forgetting is a part of forgiveness or healing? Eidetic memory seems fraught to me, but I'd love to know your thoughts on this.
> We record anything you’ve seen, said, or heard and make it searchable. For your privacy, we store all of the recordings locally on your Mac. Only you have access to them
Until an unauthorized person gains access to your computer.
I'm assuming there is some sort of encryption, but if the software can transparently decrypt the data to access it, then what is to prevent someone stealing the key from the software?
How is this affected by wiretap and privacy laws, where you would need the consent of other people in the room?
At home, I certainly don't want something recording every sound.
Even at work, I don't want everything recorded. Sometimes people need to communicate freely and discretely, e.g. your manager says to you like "Don't prioritize the CEOs request. It's a bad idea and won't go anywhere. If he asks you, we're working on it. I'll deal with the fallout later."
> I'm assuming there is some sort of encryption, but if the software can transparently decrypt the data to access it, then what is to prevent someone stealing the key from the software?
You are right that if someone compromises your physical computer and knows your local Mac user account password then encryption won't help.
> How is this affected by wiretap and privacy laws, where you would need the consent of other people in the room?
While the laws differ from state to state, we believe privacy is so important that we recommend users of our product hold themselves accountable to a much higher standard than the bare legal minimum. We believe all users of our product should proactively seek consent from everyone they record even if they are not legally obligated to do so. Not only is this is the safest approach legally, but it is also just the right thing to do.
> At home, I certainly don't want something recording every sound.
We don't record every sound. We prompt you to record audio each time a new Zoom meeting starts. You can choose to record or not to record. You can also pause at any time and delete anything retroactively.
1. You claim that the user data is stored locally, that no data is sent to the cloud but audio is sent to a cloud transcription service. How do you reconcile these things?
Given that OpenAI Whisper is open source now, and pretty near SOTA, I think creating an audio only open-source version of this shouldn't be difficult. However, I don't know how to easily contextualise the audio - how would I search 'name of the movie I was discussing with Zeynep last week'?
Yes it does seem within reach. Even just the base-en whisper modal with 74M params performs remarkably well on transcription (the large model has 1550M params!).
Compare a base-en whisper transcription to a human transcription. This is the A Powerful Theory of Why the Far Right Is Thriving Across the Globe latest episode of the Ezra Klein Show, transcribed just now:
I got OpenAI Whisper running locally on my Mac but the plumbing to make it NOT tax system resources (like CPU) and to get it to work with search isn't trivial. It's on our roadmap.
You might find my inference implementation of Whisper useful [0]. It has a C-style API that allows for easy integration in other projects and you can control how many CPU threads to be used during the processing.
The problem I have with this is that meetings and discussions—exactly the kind of thing you would want to recall—involve more parties than just yourself. But rewind appears designed to record everything in the background and be left on, not enabled manually when you want it. That means in every conversation you have you’re either going to need to get consent to record (which is super weird for 90% of quick chats) or just do it without consent which is creepy and probably illegal in a lot of places.
Well, if things don't work out in this use-case, there's always an opportunity to greatly evolve the tech industry with your compression. That resultant comprssion size sounds crazy amazing! Kudos to you and your team for your launch!
Great concept! Love the time machine inspired UI for rewinding. And super-curious about the compression. My guess is it's by saving diffs/incremental changes to the screenshots using Computer Vision.
IMO, this only accelerates the problem of memory loss. The phenomenon is known as "digital dementia" or "digital amnesia". If you want to strengthen your memory, use less technology to remember things for you. If you already have memory problems, and need aid, I can see this being useful.
I've started keeping a mood journal after a mood swing caused a very negative incident last year. It's been absolutely indispensable.
I could have used "less tech" and kept it in notebooks, but (a) handwriting is time-consuming, (b) handwriting a lot is fatiguing, and (c) I could lose the notebooks (like I lost the diary I kept when I was younger or the notes I exchanged with my first girlfriend that went into detail about what I did that day, both of which I wish I had back).
Using an app to keep this journal has made it really easy to correlate my mood to external events. Since my mood is strongly correlated to the amount of sleep I get, I can't wait to stitch my sleep and calorie consumption data from Health with my mood journal entries.
Ironically, the journal has helped me remember things I would have long forgotten!
I don’t know if this is in your blog anywhere because I have not checked yet, but what are you using for your journal app or software needs now? Do you feel the CBT helpers found in many popular apps are necessary?
I'm using Daylio for my mood journal. It's private. I use it every day, sometimes multiple times per day. (I _always_ use it when a significant emotional event happens.)
I love it because I can explicitly pick my mood based on five options (rad, good, meh, bad, awful), can create and pick unlimited additional identifiers to associate with each journal entry, and can see reports showing me not only my average mood for the month/year but also which identifiers corresponded to what mood.
Example. I slept well today, felt really good, wrote code, spent time with my wife and worked out. So in my entry today I picked the "Good" mood (internally a 4/5), then picked the "good sleep", "home", "sunny", "time together", and "wrote code" identifiers. I also have a custom identifier called "do i need alone time" wherein I can pick how badly I need it on a scale of 0-2. I didn't feel like I needed time alone, so I picked "0". Then I wrote my entry.
At the end of the month, Daylio will send a push asking me if I want to see how I did this month. In that report, I'll be able to see a line graph of my mood along with the number of times I selected identifiers like "good sleep" or "time together", in descending order.
It would be REALLY powerful to connect these entries to Apple Health, as I track my sleep and what I eat. This would answer questions like "how does my mood change when I cut carbs to 150g/day max and 1800 cals/day max? how long does it take for changes to set in?" or "how does my mood affect how much I can push it in the gym, or vice versa?"
For my work journal, I track that in Git and have Alfred shortcuts to make that easy.
> It would be REALLY powerful to connect these entries to Apple Health, as I track my sleep and what I eat. This would answer questions like "how does my mood change when I cut carbs to 150g/day max and 1800 cals/day max? how long does it take for changes to set in?" or "how does my mood affect how much I can push it in the gym, or vice versa?"
Aye captain - tis what I'm working on. We should zoom sometime.
The key enabling technology that makes Rewind possible is Apple Silicon [0]. We utilize virtually every part of the System on a Chip (SoC) so that running Rewind doesn’t tax system resources (like CPU and memory) while it is recording. It feels virtually imperceptible.
I would love a better understanding of this… while I have a an appreciation for metal and the frameworks offered I’m confused how low level you can go when A-Linux had such issues getting going - abs further, the API’s natively available could actually be private? (Which would make sense why it would only work on Mac and not iOS as bypassing the App Store as possible)
Not to be a FOSS-thumping loser, but an a16z backed company that records anything and everything on your screen sounds like a recipe for disaster. May as well set the countdown for the bait-and-switch.
Looks like a cool demo with neat tech though. Maybe someone will reimplement as OSS?
I think a better title would include <Digitized> before <Life>.
The problem with relying too much on things like photos to recall your memories of previous events is that typically the photos overwrite the actual memories, i.e. you lose your built-in record of sensory impressions of events.
Used sparingly, it's probably okay, but working on improving your built-in memory is also a good idea (i.e. the whole memory palace approach).
This kind of thing could be a great aid for early-stage Alzheimer's patients, however.
As far as being used for mass surveillance and social control by authoritarian states bent on managing their captive populations in some dystopian social-credit-scored plantation-labor system, well, I imagine the NSA's DARPA-sourced 'Total Information Awareness" programs are well ahead of anything seen in public by now.
Overall it has helped me in a couple situations. I will say when I was onboarded I don't believe audio was mentioned and a quick search for things I know I said in Zoom recently comes up empty--perhaps that functionality is new?
For now, I've found that I haven't yet hit the equilibrium point where I've forgotten something badly enough to replace my instinct to use Spotlight, browse Finder, or even search Google Drive (shudders). The Fitness app on iOS needs 180 days of data before it shows you trends; I feel like Rewind is similar in that I need a critical mass of data and time before I truly don't know where to look and go to Rewind first. Audio support would greatly reduce that window, however, so maybe I am missing out.
Overall I definitely recommend trying it out, however!