They're listening, but not to your voice or with a microphone.
Facebook is listening to your data--to all of our data, all at once. They have locations, searches, clicks, messaging, photos, hashtags, and any other form of browsing patterns for everyone in your country, everyone in your neighboorhood, everyone in the same room as you.
They have enough data with such advanced analysis that on occasion they can get really close what you're thinking/doing without you explicitly telling them. They're doing this frequently enough to creep a lot of people out and they're only going to get better at it over time.
What we have here is something like Turing test for privacy: a sufficiently advanced amount of data and analysis will be indistinguishable from surveillance.
It's simply unnecessary to listen with a microphone.
Edit: found this slide from F8 in 2015 where they said they store 300PB and process 10PB (with a P) of data per day https://www.instagram.com/p/0tUjrQKH6R
Yes, I think people really underestimate how much data they leak all day long, and how strong the pattern is. Simply emitting bluetooth and wifi, nevermind cellular, all day long can expose an incredible amount of stuff, provided someone can piece it all together (which is exactly what Facebook and other ad companies do).
We like to pretend we're unique and powered by free-will but the reality is most things you do are fairly easy to track and predict.
We're not in 1984, but certainly the capability is _already_ there. The difference between us and 1984 is only really one of intent, and you can guess how long that'll last.
> The difference between us and 1984 is only really one of intent
I'm not convinced intent isn't there; I'm also confused why'd you assume that. As a free software activist, I wonder have you every checked EFF or FSF websites. I'm not saying EFF and FSF are saving the world (no, they are not) but if you browse what they're dealing with, you can easily see the intent is there. It is simply hard to do all these without people thinking intent is not there, which is why nothing is done as explicitly as 1984. For example, Windows 10 has built-in ads, INSIDE operating system, your OS, which you already paid quite a price, forces you see ads; and tracks your behavior "to optimize user experience". People are not worried about this since ads are such an integral part of our lives. But Microsoft perfectly intents to track which websites I visit, what do I eat, what do I like, where would I go. Maybe you can say intention is "different" as in in 1984 they use my personal information to prosecute me, whereas this is not what Microsoft is after. Well, then this brings us to some Foucoult reading. In a world where all my information collected what is the difference between using all my information for a specific cause, when this cause can change just tomorrow. What if NSA asks Microsoft to give bunch of people's data? (we all know this already happened, but it is better practice to ask questions)
I am a programmer, I use my computer all the time, pretty much every single awake moment of mine. I don't want to be watched. But as you said it is depressingly hard to achieve this, since it feels like only 0.1% of the population shares this same desire. Most people are okay being watched, and this is part of the problem.
I'll skip the futurama reference for now, but for me ads are far rarer than in the 90s. Adblock handles most things online, iPlayer, Netflix and amazon don't really have adverts on TV (they have a graphical menu but I would call that an advert), Spotify doesn't have adverts. Sure there are a few adverts if I go to somewhere like london, and Sainsbury's advertise their own deals in store, and of course products on the shelf are adverts in of themselves, but the only advert I have seen in the last 24 hours was one raising awareness of cistic fibrosis in a billboard outside of Sainsbury's. Oh and the train announcement saying "the on board shop is open for teas coffees and light refreshments". There wouldn't have been any product placement in the episode of once upon a time I watched last night either.
The TV jingles I remember from childhood are still etched in my brain, the adverts I see in this decade are ones I hunt out depberatly (the John Lewis Xmas advert for example, or film trailers, or something about amazon delivery people stealing your sofa that was recommended at work)
I did see adverts on Saturday night when we went to see Thor. I don't recall what was advertised other than odeon limitless, the trailers are adverts too of course, alas they don't really work - we'd rather see more films at the cinema than we do now, but getting babysitting is a pain. We missed Spider-Man and kingsmen for example.
If anything there are more opportunities to avoid adverts than ever before.
It definitely depends where you are living, and how you travel.
If you haven't been to London recently I understand you might say there are a few ads, but lately I have really begun to notice just how prevalent they are. The tube is absolutely plastered with them - in the train carriages, the escalators, the tunnels between platforms. Construction work fences are covered in ads. And this is for thousands of people who travel 30-45mins two times every day, completely filled with ads. You don't necessarily pay them much attention every day, but they definitely have an effect on which companies, brands and even theatre performances pop into my head first.
On top of this, I took a flight to Heathrow last night and there were ads on the plane, ads all over the arrivals area as you walk to the taxi rank. In the Uber home I noticed the motorway connecting Heathrow and London has bright electronic billboards one after the other - it never seems to end.
I just wanted to give some perspective on what the commuter experience is like in London (and I'm sure in other large cities, too). It may feel like you experience less ads personally, but that may not be representative of the general public.
The apps you mentioned are a select few (admittedly good examples of how TV ads have gone down) , but most free apps now rely on some kind of advertising model to be profitable. Almost every social media platform forces ads into your timeline. Perhaps we just don't tend to notice how pervasive they are.
I did notice adverts on a flight from delhi last month on the front of a film, but it's very rare I'll watch anything on a plane's entertainment system, partly because of things like adverts.
Not sure I agree r; Netflix and Spotify. Both have curated home pages with questionable amounts of accuracy. Netflix in particular, places all of their own content on the top row of the selection. My fire tv stick home page is jammed with ads for content on the landing screen. Amazons website is a gigantic ad. My online shopping now has suggested items based on what I’m ordering( ordering strawberries - now you get suggestions for branded cream in the same search)
Product placement also appears to be rampant (despite you mentioning it explicitly in your post) - many shows highlight specific vehicles and brands, or specific phones, brands of food...
Another one I’ve found is lots of blogs (particularly food ones) have sponsored entries. So you have someone writing about food ad having recipes that use a particular brand of harissa, or a particular food processor.
All these things are advertising, and are mostly heavily targeted too.
Quick note to say that despite umatrix/ublock origin, it's hard to not be tracked via canvas fingerprint tracking, where the use of a blocker can make you more trackable. I mean it's not necessarily the ad companies at that point, but you're still a data point for marketing/surveillance.
> Most people are okay being watched, and this is part of the problem.
I am not sure I agree with that. I think the vast majority of people react to the same way to surveillance, they at most don't like it, or outright hate it.
But most people are not conscious of this surveillance. If there was a guy following them absolutely everywhere, all day long, and logging on a notepad everything they do and everything they say, they would go absolutely mad. It's just that current surveillance, through cookies, CCTVs, server side logging, is too stealth for people to notice.
Implies only one can be right. There doesn't have to be only one answer.
Some say the world will end in fire,
Some say in ice.
From what I’ve tasted of desire
I hold with those who favor fire.
But if it had to perish twice,
I think I know enough of hate
To say that for destruction ice
Is also great
And would suffice.
Everytime we avoid one possible dystopia, the possibility for another will surface. We just have to hold them at Bay faster than they can arrive
> "that Huxley not Orwell was right" Implies only one can be right.
You're very right about this observation. That being said, I'm kind of tired of the 1984 meme because the state of things and the way we seem to be heading looks nothing like the future presented in 1984. In fact, if we are expecting dystopia to present itself like the one portrayed in 1984 it might obscure the fundamental issues and complexities that our society faces. Brave New World, on the other hand, has a better chance of helping us reveal them.
When a person submits their email for free Wifi at a place with Zenreach, they pull from databases and build a profile on the person.
I don't know which services provide this or how they gather data, but the thought of anyone being able to download a detailed file on you with only an email or phone number is scary.
A few months ago I did a Google search for my name as i do once or twice a year just to see what comes out. This time I had mixed reactions because among first results there was full text taken from a scanned old magazine dated 1978 or so with my name complete with street address. That was long before the Internet was publicly available and having such data into a magazine would not hurt anyone because people expected it could be useful to get in touch with others with similar interests (electronics in this case), and it surely worked for me because I got a free subscription and some job offers which I would have gladly accepted if I wasn't only 12 years old:)
But nearly 40 years have passed, the ubiquitous magazine called the Internet is read by anyone for free, and having your address and/or personal data there isn't that safe anymore. Luckily I relocated a few times since, but what if I didn't? Technically speaking it's great to see an old magazine brought back to life, but what about unfiltered personal information one would expect to remain buried that will instead remain available forever in other contexts?
I'm not bashing Google's bots crawling around to turn anything into searchable data, and I surely would never ever want laws to limit what can be searched, but pay attention if you shared personal data on printed material because if until yesterday we said "everything you put on the internet stays there forever", today there is more.
As a company director since the age of 15 I’ve had my personal address publicly listed for quite a while now.
Prior to that there were phone books and whilst you could choose to be delisted clearly a lot of people didn’t since the books were so thick!
I thoroughly agree that privacy is important but I don’t think people ever really minded very much or if they did not enough to do something about it like lobby for laws to ensure personal addresses are not disclosed publicly.
What's the reason for keeping your address a secret? Genuinely curious, as where I live (se) almost all peoples addresses are public information, and it's really practical.
It takes only 33 bits of distinctive information to identify a given person. Specific information about a person, including background, can help provide further information on them -- it tells you where to look (and more importantly, a very good idea as to where not to), who to talk to, and what they might have done.
"If one would give me six lines written by the hand of the most honest man, I would find something in them to have him hanged."
- Cardinal Richelieu (a/k/a Armand Jean du Plessis, Cardinal-Duc de Richelieu et de Fronsac)
"the forced revelation of information makes individual privilege and power more important. When everyone has to play with their cards on the table, so to speak, then people who feel like they can be themselves without consequence do so freely -- these generally being people with support groups of like-minded people, and who are neither economically nor physically vulnerable. People who are more vulnerable to consequences use concealment as a method of protection: it makes it possible to speak freely about controversial subjects, or even about any subjects, without fear of harassment."
> What's the reason for keeping your address a secret?
I, and the people I love, have public political opinions on the internet, and SWATting is a thing.
I can either keep my address a secret (and I do; I use my PO Box whenever possible), or I can decline to participate in public civic discourse and encourage the people I love to do the same.
I think it's not so much keeping this sort of information a secret, it has never really been a secret as anyone with some motivation could find out these things.
The big thing these days with computerisation and the Internet is it is making these things trivial to acquire, accumulate, and search. It's not a matter of going to offices and looking through paper records, because it's all at someone's fingertips.
We _are_ in 1984, but unlike the book, it's not imposed on us. We signed up for it. And those of frequenting HN most likely are the ones helping to build it.
Think about it: tomorrow is Monday. How many folks on this board will be working on analytics and tracking once they get in to the office? A lot of us, guaranteed. And how many of us will take a principled stand, swearing not to work on products until surveillance features are removed? A few of us, guaranteed.
You say people will take a stand, but the vast and overwhelming number of us wont, we will just continue to help build the greatest surveillance system in history. And then tell each other how we’re unicorns that are saving the world, that we cherish freedom and liberty and the hacker ethis, when really we just want to get paid.
Nobody knows what stand to take. We stood and fought for open source everywhere, thinking that would be our salvation. But apperently there is more to it than that, and we aren't exactly sure what.
Sure we are sure, we just don't like it. RMS gets a lot of flack, but he's mostly right---this is not a technical problem with a technical solution, it's a political problem with a necessarily political solution. Most of us live in democracies, so if you want to fix the problem, it's actually quite straight-forward, it's just that people don't want to give up their nice job and nice hobbies.
Oh my god stop being so dramatic, both of you. The vast majority of people are not actively contritubing to the sorry state of data privacy nor are they heroically working to fix it. Put the blame where its due.
Right, like the example from Target. It is possible that facebook has access to a stream of Target purchases or credit card purchase information. I'd be 0% surprised if there was a data sharing agreement with credit rating companies.
Even if they just knew the price of the purchase, combined with the job (burn risk) and location data (traveled from work to nearest store with pharmacy during the day), it's possible that you might be able to infer a burn with enough accuracy to be valuable to advertisers.
"It is possible that facebook has access to a stream of Target purchases" ...
It's not possible, it's simply true. In fact, this feature of the facebook ad platform is available to any advertiser on Facebook/Instagram/etc, whether you have $100 to spend or $100M.
You simply upload your customer's purchases to Facebook with data such as zip code, email, etc etc which Facebook then uses to optimize your advertising budget and find more customers similar to the ones you already have (the feature is called look-alike audiences).
I'm sure Facebook would like to do what you're saying, but I think that they're a long way from having those capabilities.
They have billions of users, billions of ads, and trillions of pieces of metadata. Sifting through that to produce a guess like "User 1234 burned themselves and would be interested in product 5678" would be an amazing and scary piece of AI.
Not to mention that Facebook are limited by the ads they can sell. The manufacturer of the burn cream doesn't give Facebook a bucket of money and free reign. They choose how and where the ads are shown. There is currently no option for "Users who recently burned themselves".
Last year there were 29,000 categories that could be targeted. If an ad sales rep wanted to close a deal, how difficult would it be to add one more for recent burn victims, or people who recently bought a specific product?
From the links that m_ke posted, facebook already has ties to loyalty card information, so it's very possible that they didn't need to do any inference, they just had the data directly.
In any case, the main point is that facebook doesn't need to listen to what people are saying, it has a ton of other data streams that could explain the stories in the link.
What would the point be in advertising a burn cream to someone that just bought a burn cream? It's not like it's a common reoccurring purchase (like Toothpaste for example).
Like the article says, it's probably just the Baader-Meinhof phenomenon* at work. It's like when you buy a new car - you suddenly start seeing that same model of car everywhere.
I think the proposed logic here is "a person like you buys burn cream so we advertise burn cream to people like you." In this scenario the person who actually bought the item will get the ad too.
To me, advertising a product to people who already bought it is a sound strategy. I see no contradiction. I'd rather like to hear a cogent argument as to why that is wasted effort, as is usually implied.
In your example, maybe I left the bottle of burn cream at the office. So I will want to get another one on the way home. It's effective to get reminded in my Facebook about that item (and the brand) isn't it? Yes it's creepy but it works.
> facebook doesn't need to listen to what people are saying, it has a ton of other data streams
The same could be said for any of those data streams. But just because Big F has access to those other sources of data doesn't mean it excludes voice recognition.
I'd be very surprised if FB wasn't in this arena. That's how big tech companies work these days — they do what everyone else is doing. Like getting into self-driving cars.
Apple[1], Google, Amazon, MS, etc... all have voice recognition boxes. FB doesn't need to market a box for your living room, it already has one in everyone's pocket.
([1] Apple says it doesn't use any of your Siri words against you. If you don't trust FB's word, you might no trust Apple's, either. But at least on the Mac you can keep dictation off the internet, if you choose.)
I agree that even with an enormous amount of data there are limits to what facebook can realistically infer...I particularly enjoyed the things that facebook’s ad platform determined to be my hobbies when I checked recently: https://m.imgur.com/nWCWn63
I’ve had an account since 2005, even if I use it much less than I used to, you’d think they’d have enough data to do better than that.
Think smaller. They don't need to consider billion users at once, for modeling you can start with hundreds of thousands or few million. Trillion pieces of metadata can be reduced to tens of thousands of features (eg. instead of feeding in the raw status updates, assign each update to some class, put in some mood measurement etc). This reduction does not need to be fully automated, it can be human assisted (humans coming up with ideas on what kind of features to extract). Instead of considering all possible ads, you can start by taking out the long tail of ads with very little interaction data and focus on some specific categories.
Building this kind of predictive models is not black magic. This is what companies do to figure out who they should target on direct marketing campaigns. Applying some magical deep learning dust can quite likely improve the results (in Facebook scale), but it's not mandatory. Computationally the hardest part is building the models. Once they are done, applying them to millions and millions of customers is more straight forward.
When considering what is possible, you need to also consider the stakes. Facebook ad revenue is something like $10 billion per quarter[1]. To make more money, I can think of three ways: add more users, make users spend more time looking at their feed or make more money per ad slot. Better targeting means more money per ad slot since some customers are paying directly based on the clicks. Since improvements lead to better revenue, extra hardware costs are easy to justify.
Also, Googling something on your phone is so common and easy nowadays that people just forget that they did it. It's literally become an integral part of the thinking process.
Rewards cards / loyalty card are designed to incentivize you to spend more with the stores. Ostensibly reselling data is a lot less valuable than directly trying to convince you to shop more at their establishments
That was true at the beginning of the loyalty card phage. But now the big chains are learning how to data mine and discovering the value of their data to others.
Now your loyalty card data is worth twice as much. Once to the retailer for internal marketing, targeting, purchasing, etc... and again selling it to outside parties.
thanks, sorry to reply to this late. I often wonder if I take a loyalty card and then for example I purchase twice as much double cream as the next person, does this get sold to health insurers who then charge me a higher premium? The permutations are endless eg too much beer. too many condoms. Too much spicy dip.
If you look at the history of the Tesco Clubcard (one of the early ones to appear in the UK), the data they mined from their new ability to tie purchasing trends to customers more than made up for the cost of the rewards being offered to get people to use it.
There's a reason they got so dominant over here - they were simply way ahead of the game on offering what their customers actually wanted, because they were the only people who actually had a clue.
If you read the Walgreens Rewards privacy policy, it covers everything from retail transactions to security camera footage and they are able to tie everything together. They're also able to gather and sell data about your health, which would otherwise be protected by regulation using the rewards program, especially as people are incentivized to use the card, even for prescriptions...and no one actually reads [between the lines] of the privacy policy
That's true and credit card companies are very open about it (at least if you read their reports for investors). The advantage of store cards is that the store knows exactly which products you bought, whereas they only know the total if you used a Mastercard/Visa.
>whereas they only know the total if you used a Mastercard/Visa
This is simply not true with respect to the big retailers in the US. This started as far back as '06, though the details of the implementation have changed a bit.
Only if it's a co-branded card. If you use your Visa issued by some bank to go shopping at Walmart, they won't get your personal details together with all items you bought. They can get the items but without the personal details (including age, wealth, etc), they're not that useful.
We had the ability to do this at the largest merchant acquirer 10 years ago as long as your bank signed up. Walmart was one of the first customers and would stream SKU-level data straight to our servers from the POS terminal.
You make a very good point. However, in this case there is very strong evidence that Facebook is also listening to the microphone.
Coincidentally, I recently had a friend convey a story where the Facebook app suddenly recommended a new connection immediately after certain information was spoken in a verbal conversation. This instance was particularly damning because, due to the sensitivity of the information that was spoken, it had been very carefully kept out of the digital footprint.
I consider it likely that this is merely just another example of people underestimating how much data they're leaking. I would wager to guess that there would have been far easier methods of deducing that new connection other than listening to your conversation and somehow inferring it from that stream of noise. You just don't know which, but it's not unimaginable that that certain information was leaked in another way as well, and probably in a more structured format than unlabelled audio.
Once Facebook recommended me to connect to guy with an interesting name, and I wondered where I saw this name before. I looked up my emails, and I saw that I bought something from him on eBay several years earlier. I know I never gave Facebook access to my mail account, but guess what, I'm pretty sure the guy gave access to his. Facebook saw that we sent mail to each other, and it asked me if I wanted to be FB friends with this random eBayer...
It's not unimaginable, but the frequency of these stories should make you wonder.
I had this happen personally. Had a conversation about my work with this girl I know which ended up being mostly about project management. The girl told me later she started getting ads for project management stuff.
It's not a subject she's interested in, has ever searched for information on, and has no relevance to her job working the counter at a sandwich shop. We have no social media accounts in common and don't even talk that often.
The issue is in the story teller. If the person telling the story doesn't understand what other data they're leaking or the data the people they interact with are leaking you cannot take their word that "Facebook is listening to my microphone" at face value, no matter how frequently they say it.
Also wouldn't a constant stream of audio - even low quality audio - ruin battery life? I realise it's a phone so it's base usage is a constant stream of audio but I can't help but feel that or something else would be giving it away.
As they have a bug bounty program I imagine there's plenty of people watching raw network activity between app and Facebook too.
Facebook probably knew because she was logged in to it on her work computer, where she was also searching for and reading project management websites. Since every website has a like button it means that Facebook can correlate the two.
This girl has zero interest in project management, the whole reason she brought it up to me is because she thought it was weird because it's a subject she never even thought of outside of the one conversation we had just prior to her being fed ads for the same thing.
Exactly. People underestimate the amount of data Facebook (and companies alike) can and do scoop up. I’ve seen countless stories supposedly proofing that they’re listening through the microphone. However, most of them can easily be explained away if you consider the vasts amount of data they collect.
The idea that they're listening to your mic is rather ridiculous. I mean, we'd find out eventually, and the consequences of widespread illegal wiretapping would be severe for Facebook, they would really cease to exist.
However, as you said, they don't need to. Localization and usage metrics alone can tell you an incredible amount of extremely detailed information.
Just looking at the Facebook iOS app, they don't declare the "audio" background mode in their Info.plist. So instantly we know that they're not recording anything in the background.
For foreground, I suppose it'd be fairly trivial to tell if the app was making calls to AVAudioRecorder.
I wish someone would actually write about how ridiculous these clames are in a way that non-technical people would be able to understand. There's so much bias, conjecture and downright false proof.
I agree. I'd love something shorter than having send someone to read half of the LessWrong Sequences. I wish CGP Grey or Kurzgesagt could do a video on that.
The basic thing you need to make people comprehend: to learn a fact X, you don't have to actually learn it - it's enough for you to learn about such Y that P(X|Y) >> P(X). And you don't have to learn Y either, you have to learn V and W such that P(Y|W) >> P(Y) and P(Y|Z) >> P(Y). Etc. This method applies recursively.
And once they comprehend the causality graph this forms, you need to make them understand just how much information we radiate all the time, and how humans are still very inefficient in doing the calculations mentioned above with all that data. Facebook is only the tip of an iceberg; it's only going to get worse from here, because modern technology keeps letting us explore the causality graph even further and faster.
> the consequences of widespread illegal wiretapping would be severe for Facebook, they would really cease to exist.
Unfortunately, I don't think they would. They'd get a slap on the wrist fine, maybe have one or two senior executives resign, they'd post a mea culpa, saying how they now it was wrong and promise to do better, and then everyone would forget about it a few weeks later when some new thing started to dominate the news cycle.
Indeed. I doubt they would go down due to this. They'd receive some negative attention. They'll make some sweet PR statements and hop, they're back in business.
It's a pretty easy test - they just said "cat food" a bunch of times and ads for cat food came up in their feed the next day. I'm sure if you were interested you could probably do the same experiment with some other product you've never posted about or talked about online and come up with similar results. It seems strange that you would just assume that the people who are bringing up this issue are simply misinformed, or do not also know that facebook also looks at other parts of their data. I think most lay people understand how data mining is leveraged these days.
I'd like to point out that this video doesn't actually provide any evidence that Facebook was listening.
They said they talked about cat food all day, then they showed an ad on Facebook for cat food.
There's no way to verify that they didn't make the video of the cat food ad on Facebook and then talk about cat food all day.
Nor is there any evidence that they talked about cat food all day at all. No recording of the conversations etc.
In fact I'm a little confused about how this is even a question.
I'm not a programmer so I don't know, but: Wouldn't it be trivial for someone who knows how to write Android software to monitor if an an application is accessing the audio input device?
I mean, I know that on Linux you can monitor whether or not a device is being opened.
Why doesn't someone check if Facebook is accessing the audio input?
That's been done, and no one has found any evidence that this is happening. Hence it's status as a conspiracy theory.
As soon as you start assuming that the OS is giving the facebook app access to the microphone regardless of whether you allowed it in the settings, things start to get absurd
> I'm not a programmer so I don't know, but: Wouldn't it be trivial for someone who knows how to write Android software to monitor if an an application is accessing the audio input device?
Or even by MITMing the connection and looking at the network packets. But yes, it's not outside the realms of possibility to hook into the microphone driver on a rooted Android phone and check when it's being activated.
I'm not one to leap to Facebook's defence and if this is happening it needs to be shut down ASAP, but I suspect that there would be at least some credible evidence out there if it were indeed the case.
I'd like to see some more rigorous testing than this. They said it took 2 days to happen? Other people's anecdotes indicate it happens within minutes. I don't think this is conclusive, it's still within the realm of confirmation bias.
I feel like there'd be hundreds of these videos if it was real. How many people tried to make this video and it didn't work? I hope someone buys 10 new phones, opens 10 new Facebook accounts, and does some statistically rigorous testing.
This is easy also to fake, and I trust some random YouTuber much less than a public Facebook statement. It would be intensely stupid for the head of ads at to deny this if they were doing it.
Well, she wasn't under oath, so there is no firm legal requirement for her to be truthful. Or maybe, being a public representative, she's not been briefed about this. Legal security through compartmentalization is pretty common in the corporate world.
That could easily have been a random ad, combined with something like the Baader-Meinhof phenomenon.
To be a scientific test it needs a list of products, and then randomly assign half into a study group and half in to a control group. Then talk all the time about stuff in the study group but not the control group. And see if there is a statistically significant difference between the ads shown for products in the study group vs the control group.
IN the video he claims not to have owned a cat or talked about cats for 20 years. they chose a subject that they are certain they have, as a couple, never exchanged words on.
Explaining away doesn't explain. For an average, normal FB user, this was as scientific as they could get. It would be nice to see someone repeat the experiment with a pro-hacker type on hand and a packet sniffer ++.
There are too many people giving a company like fb, who have form on the implementation of morally and ethically dubious practices, the benefit of the doubt, all the while dismissing any and all claims of users. It looks fishy at best, and I'm not name calling, it isn't allowed on this forum, but there are a disproportionate number of FB defenders popping up wherever this story surfaces.
Except he probably sees hundreds of ads for things he's never talked about for 20 years. It's not scientific at all, because he has no way of distinguishing a listening ad from luck. My statistical method would distinguish that.
But I guarantee you there are hundreds of pro-hacker types not giving facebook the benefit of the doubt. They're reverse engineering the apps, monitoring API calls from rooted phones, monitoring network traffic, etc. There are entire forums of people dedicated to hacking on android and finding rooting methods, etc.
There are tons of security researchers in academia desperate for a nice paper, eagerly looking for something juicy like facebook app listening to users. In fact these researchers have found that hundreds of android apps are listening to you[1]. But facebook isn't on that list.
The EFF has tons of smart people eager to dig into any little privacy mistake that a major company makes. They for sure would launch a huge lawsuit against facebook if this came out, and they very likely have some researchers looking into whether there's any listening.
Why haven't you heard about all the pro-hacker research into facebook's spying? Because they have all found no spying, because there isn't any. They don't want to publish that they found nothing, because then it would look like they're defending facebook, and these pro-hacker types don't like facebook and don't want to defend them.
> Facebook is listening to your data--to all of our data, all at once.
So is Google. What makes FB even creepier is that they don't just watch everything people do, but they manipulate people in controlled experiments to extract as much money from them as possible.
This is the thing. It's a fine line between offering someone a solution they might want or appreciate based on their search. Another to manipulate them into wanting it through emotional plays.
> and any other form of browsing patterns for everyone in your country
Definitely. Basically every major website has facebook code on it for social sharing and for "analytics". Facebook surely uses this analytics data to track you from site to site and know exactly what you're looking at. If your browser is signed into facebook, they know exactly who you are too.
That's all very true and very important, but I think it misses the point of contention that many people are rightly focusing on.
There's an issue of whether facebook is, despite what they say, doing some sort of data analysis that's relevant to advertising, in addition to all the other listening and analyzing they do with the rest of our data.
What an insightful comment. Not sure if that’s your idea or not but the bit about enough data being indistinguishable from surveillance is very interesting and if evolved properly may one day make a great argument for constitutional protections around data collection and privacy.
The number of reasons you should care are numerous. One facet of the dystopian nightmare that total surveillance inevitably brings is already being realized in China.
They're collecting data on children without their informed consent, babies today have their information capital compromised from the moment they are born, via their parents shares, purchases, and activity, without their say.
Because the risks involved with complete knowledge of everyone at every moment are huge. Consider that most developed countries have for decades spent millions in spy agencies to get just enough dirt on people of interest to be able to manipulate them.
Imagine that the same information is now available on all people to many giant companies, some with almost government like spheres of influence, you can see the potential for manipulation grow.
The other part of Facebook in particular is that they sell that leverage to advertisers.
An example of that would be the possibility that Russia influenced the US election. Whether or not it happened in either direction, the concept and possibility is something to be worried about.
The reason we should still care even though it seems ubiquitous already is because we messed up by getting here, but we can still fix it.
Already many companies check the social media accounts of people applying for a job. A company can't legally discriminate against you based on gender, race, ethnicity, age, etc... But an HR drone can easily roundfile your resume because he doesn't agree with your political views.
That's why employers requiring social media information is illegal in a dozen states or so. (Illinois, I know for sure. There are others.)
Facebook is amassing a huge amount of valuable information on every member.
I disagree that the threat of outrage is sufficient to stop the information being used badly. For example, there is outrage against Equifax, but their cache of information has still been compromised. We have seen data leak after data leak from various companies. If a country's spy agencies want to go after the data, they have a lot of resources, from hacking to legislation to physical intrusion or coercion.
Add to that, it seems like Facebook doesn't institutionally care about privacy (probably because it is hard to explain something to someone whose paycheck depends on them not understanding it). For example, http://actualfacebookgraphsearches.tumblr.com has some very damaging graph searches. Or people who have been outed as gay by incomprehensible privacy settings.
Facebook is a sieve, and the reason to care about them having a lot of information is the same reason to care about privacy in general.
Remember when Target snitched to a teenager’s parents that she was pregnant because her purchases fit the “is pregnant” profile, and everyone got outraged, and they stopped profiling people?
Actually, only one of those things happened, the one in the middle. I'll let you dig into Forbes's "sources" to figure out how they managed to twist a hypothetical into something that already happened.
The author talked to an employee who talked to the man whose daughter was flagged as pregnant. It would be nice if they could have talked directly to the man, but that doesn't seem too distant. In any case, it hardly seems to warrant a scare-quoted "sources," nor would I describe that as "twist a hypothetical."
You have now refuted the story twice in this thread without providing any evidence of its inaccuracies. While I don’t know for certain the validity of the story, you aren’t doing much to persuade me of your position.
I ... don’t know what to say. What they are doing now is respectful?
If we ever come to widespread outrage, it will be much to late. In fact, I don’t think there can be outrage. FB would detect it and make it disappear. Somehow.
For me the fear is insurance companies denying cover or increasing premiums based on some random search, share or like that actually has no relevance to the cover, then not telling you why they made that decision.
> "At work, happened to me though earlier this year. Working as a barista, got a burn, talked to my partner in person about it, went to Target, bought the burn cream, and saw an ad on FB for the exact product I purchased. Never searched for product either," wrote Brigitte Bonasoro.
Facebook has data sharing agreements with many, many companies. It's most likely that the cream was bought with a card then linked together with your facebook profile. It's not that hard, especially when you use the same email identifier for all those loyalty cards you have.
I have a similar anecdote. Girlfriend was having some symptoms, we briefly talked about it then walked to the pharmacy and bought a pregnancy test. Maybe 30 minutes after purchasing the test we watched a youtube video on her phone. The ad on the video was for the exact test we bought.
I figured that youtube got the credit card info from the purchase but I was shocked at how quickly that translated into something showing on the screen. I've always thought about these things in the back of my mind but this was still eye opening for me.
I'd never had it happen to me personally since I aggressively block all ads :)
It's certainly a technical possibility but my bets are on your girlfriend's recent searches. Was the yt video on anything to do with being pregnant?
Was the pharmacy a national chain, regional, local, or one-off?
In my experience in data exchanges with other large companies, data is usually shared and processed in batches where even an hour lag would be considered "very quick".
It's possible she had done some searches before talking to me, I'm not sure. The video was unrelated though. National chain pharmacy so they certainly have some kind of data sharing scheme.
> National chain pharmacy so they certainly have some kind of data sharing scheme.
I don't know if you can say that for sure or not.
Regardless of whom shares customer data with whom, this whole issue has a number of legal/compliance issues beginning with HIPAA. The pregnancy test purchase, when anonymized, is just a sales data point, sure. When combined with identifying your girlfriend, and then giving that data to a third party (especially for non health-care treatment or paying claims needs), is most certainly illegal. If she had purchased a Coke, then no, but given she purchased a health-related product it falls into a myriad of laws that govern health care privacy.
> I have a similar anecdote. Girlfriend was having some symptoms, we briefly talked about it then walked to the pharmacy and bought a pregnancy test.
The funny thing is, this is a horrible use of ad data. With a few exceptions (luxury cars being one, making someone feel good about their purchase) showing ads for a product someone just bought is the height of uselessness.
In a sense, they are suckers. Plus, as another story on HN suggested, their abandonment of advertising you things you don't need right now(but you may find you want later in life) proves counterproductive. A heck of a lot of things advertised to me on Facebook are things I already searched for and either purchased or already decided wasn't worth buying. I don't believe that modern internet ads are as valuable as the industry seems to think, which may be a sign of an impending bubble burst.
But still, when you advertise X to people who searched for X' (or even X itself), you can see the conversion rates of those ads, right? And you should be able to measure if that setup is profitable or not.
im bombarded with ads for Ecwid. a product i purchased and have used for 5 years. I feel bad, they must have spent more on ads than I actually pay them.
Credit card information is not tied to the products that are purchased. It is an aggregate purchase amount tied to a store location, that is it. Whatever is going on here needs to be investigated and brought to light.
No, they do not not. I have access to many billions of credit card transactions. Aggregate amount, date/time, maybe merchant location, merchant name, and a merchant classification--plus some other things but not much that is interesting.
Depends on which country you are in. Your bank card in some European countries is not allowed to receive or store info about the items you purchased, but the shop obviously will and there is a limit on what they can do with that info and for how long they can store it.
So if I pay for petrol with a card and also buy donuts and coffee the bank and the shop cannot market to me according to what I bought, but if I join a loyalty card scheme then I'm opting in and the shop can, but not the bank.
In Norway btw, where govt. are strict about this sort of thing.
> It sounds like someone is trying to game the advertiser by correlating ad views with sales.
Shhh. If advertisers realize how utterly wasted most of their advertising spending is you'll destroy about 80% of "tech" companies and plunge the world into a global recession.
To be fair, coke can't get many new customers buying their product. Yet they spend quite a bit of money advertising their product so that their existing users keep purchasing.
For example in the Automotive industry a mail piece with a tracked phone number is used and that is cross referenced with Dealer sales data. The pieces are mailed to targeted lists of people qualified by credit score, postal code and recent purchase of vehicle make. Should the site be accessed via a personalized URL the user now appears through a variety of methods, same if they come from FB they’re claimed through offline conversions where sales data is compared to audiences. Using this readily sold and available data it’s easy to calculate spend and determine response. Even with single digit response (not uncommon for email) industries thrive.
These ads also help in talking to you unconsciously, the next time a person is looking for a burn cream, its the company's product whose ad you saw will be the first thing that comes to your mind.
reinforce the quality of the product to make the buyer feel good purchasing again? odds are if someone's of a certain age/relationship status and buying a pregnancy test, they're trying to conceive and will possibly buy more.
Would you still be comfortable with it if Zuckerberg was running for president?
What ways do you think they company might be able to use that information to sway the outcome of the election (passively or actively), and do you think there is a greater than zero chance they would try to do this?
I'll sometimes see ads for products I bought at Costco, so that makes sense. But then again I get a lot of ads I don't understand -- Facebook has me pegged for a drug addict considering divorce who loves Urdu-language television, but where it came up with those ideas I don't know.
> "I run ads product at Facebook. We don't - and have never - used your microphone for ads. Just not true," Mr Goldman wrote.
Personally, I am so used to corp speak and word play by major companies that it is difficult to take anything at face value. Therefore, I think the nature of the question asked to Facebook should be different. It should be on what they do rather do they do X. So, here's my question - "What does Facebook use data gathered from user microphones across any devices for?"
In a statement issued on June 2nd [2016], Facebook said it "does not use your
phone’s microphone to inform ads or to change what you see in News Feed."
That's pretty fucking clear to me.
Original comment:
If they had said something like "We don't use microphones to gather data to drive personalisation on users profiles" there would be comments here on HN complaining about corporate speak and how they didnt just flat out deny using it for advertising.
Apps get busted for doing dodgy things all the time. The Facebook app has been decompiled and network traffic inspected and studied so often that if this actually was true then it would have been proven by now. Instead, all we have is this weird 'survivorship bias' where one time someone said something out loud there was a tangentially related ad on FB.
Today I was looking up bars in Budapest and found one on Google. I started looking at images, at the only one that loaded so far on the slow internet was of food. "Wow, Google knows me so well. All I care about right now is food and the first image it loads is of food" I said out loud to my friend. Eventually the internet caught up and the rest of the images that loaded were of food. It only showed me food photos (in a bar thats more known for its decoration and alcohol). Nothing more than a coincidence.
My question was NOT "Does Facebook use the phone's microphone to inform ads or change what we see in the News Feed?". It seems you have misunderstood my question. Let me repeat my question again :
"What does Facebook use data gathered from user microphones across any devices for?"
I understand what you're asking and you do have to be particularly aware of what /you/ are asking.
That particular question, in context of these allegations, would presuppose that they use microphone data for something other than what the user clearly is aware of, e.g. voice chat.
So an answer like "to provide users the service they need" would undeniably be spun by media to be: Facebook uses microphone to tweak news feed. The longer the answer is, the more likely it can be misunderstood and taken out of context.
You shouldn't expect a public company to answer questions like that, simply because the media will gladly take the answer out of context. You need to phrase it specifically, so they can answer without being misunderstood.
No, it's not a loaded question. It's much more like "list everything you've punched". Maybe with "in my house" appended as an analogy to it being user microphones.
The question does not presume any misuse. It just leaves no room for implications.
Apparently people think there's a secret backdoor that lets an iPhone app record audio without turning the status bar red. You could deny mic permission and people would still think it's the secret backdoor.
Well no, Apple (in an unprecedented move), granted the Uber app the com.apple.private.allow-explicit-graphics-priority entitlement.
"Apple gave us this permission because early versions of Apple Watch
were unable to adequately handle the level of map rendering in the Uber
app," an Uber representative, Melanie Ensign, told Business Insider.
"Subsequent updates to Apple Watch and our app removed this dependency,
and we're working with Apple to remove the API completely."
Uber was doing other sneaky stuff to read the devices serial number (or other unique identifiers) and evade Apple's countermeasures, but that's seperate from this.
What is the meaningful difference, in the context of tedunangst's post, between an entitlement that allows surreptitious recording of video and a backdoor that allows surreptitious recording of audio?
I'm 95% sure that there will be an entitlement for recording audio without showing the red mic bar (look at the Phone app), but it won't be a public one. There is no way for a third party app to get past app review with a custom entitlement without explicit permission from Apple. I'd be very surprised if it even passed automated review when the app was uploaded.
Uber's entitlement was always 'public'. Facebook doesn't hold any private entitlements that would allow it to record audio in the background without notifying the user.
Edit: Actually, FB will ask for microphone permission if try to record a video. Based on every other app that records video also prompts for permission to record using the microphone, I'm lead to believe the microphone permission is required on iOS if you want to record video.
FB Messenger does not prompt for Microphone until you record a video or voice memo.
Those apps have voice and video features, so it makes sense that they'd request microphone access. (And iOS lets you disable those specific features if you want.)
iOS also shows an indicator if an app is using the microphone. This is true even for built-in Apple apps like Voice Memos. There's no way to surreptitiously record.
Instagram can record video posts and Messenger can make video calls, both requiring microphone access. In either case the recording is obvious with a red bar for microphone access in the background, like all other iOS apps.
“Facebook does not use microphone input for ads, user interest assessment or for any purpose other than facilitating a single user initiated transaction.”
Its only useful to train if they also have the transcript by a human. Eavesdropping conversations and having armies of people transcribing them seems like a very expensive and illegal way to get that data when there are probably millions of available samples, TV shows, etc that have both voice and transcription available already off the shelf.
The 2017 F8 developer conference featured a lot on Machine Vision. Processing images and video for objects, such as for automatic close captioning, is where vast resources are focused. I highly recommend watching a few of the videos, they’re specially aware as well and can infer orientation of obscured things like limbs. Microsoft has real-time audio translation, doing machine transcription at scale is totally feasible.
The OP that I was replying to was insinuating that FB is collecting audio as training data to create AI models like the ones you are talking about. I was pointing out that raw audio is useless to train an AI model for recognizing words, the whole point of training data for AI is that you have an input and a known output (transcription) that you can use to train and test the model with, having just input is useless for training.
And people have a problem with it, and so the companies use Corp speak to try and make it seem like they aren't doing what you asked. The poster higher up asked how they could remove all doubt, and it's really easy. The problem is that they are trying to remove all doubt while continuing to do business as usual
That makes sense. So, given that this is data it's users have an obvious interest in keeping private, Facebook could at least inform of this. The only reason I could imagine they aren't doing this is that their voice recognition isn't used for any released products. I think that's a viable theory of what they would be doing with the voice data.
EDIT: User 'crucifiction' makes a good point about needing the transcript to use it for voice recognition. So, who knows.
It doesn't in any sense justify collecting data and being so elusive about their intent. We naturally have reason to get the impression that Facebook just wants the data, will try anything it can to keep collecting it until so much energy has been invested in the PR issues that the collection has an attributable effect on their market.
But maybe they are using the voice data to change the world for the better and bring people together yada yada! :)
>In a statement issued on June 2nd [2016], Facebook said it "does not use your phone’s microphone to inform ads or to change what you see in News Feed."
>That's pretty fucking clear to me.
Said nobody who understands law, language, or politics. It's absolutely fucking clear to everyone who reads with discrimination what this means. It means they are using your microphone to vacuum up your information. That they added the qualifier "..to inform ads or to change what you see in the News Feed" gives them the verbal and legal wiggle room they need to do absolutely anything they want with your information. For example, they could use your microphone to monitor, record and quantify everything they hear to "adjust a user profile" that they keep on all users. In that case, they could simply claim that the ads you see and the news feed you get are based on your user profile.
>How could they respond to remove all doubt they're listening?
They could very simply. "We do not listen to your microphone or record content, data, or metadata from your microphone in any circumstance". Without qualifiers of any kind.
I was about to leave an extremely similar comment, but I was going to frame it in terms of the NSA "direct access" story. I got the impression from that story that immediate denials from tech companies, which seemed categorical, turned out to hinge on which magic words were used.
In this case, when I read a sentence like "We don't use your microphone for ads" I end up doing that game where you re-read the sentence over and over and emphasize a different word each time, trying to guess which one is the magic word.
Why would they even bother? On the one hand, they don't need to, they already collect an incredible amount of data on everyone. And on the other, if they were to get caught listening, that's a gross violation of so many federal laws that it's possible Facebook wouldn't even exist the day after it was uncovered.
The risk is incredibly high, and the payoff not that great.
What we're seeing is simply that people fail to understand how much data they leak, and how effective Facebook is at putting it all together. They assume it must be magic, listening to their words, listening to their thoughts, but the reality is they don't need to, because we are _voluntarily_ telling them all this info already.
I just assume that they hire a 3rd party to do it and just incorporate the processed "external data source". Now they don't use the mic, just a text stream purchased from Acoustic Magic Analytics Inc.
I’m sure they don’t use the microphone for ads. If they use the microphone, they surely use it to feed data into their profile of you which gets used for all sorts of things... like ads. One level of indirection is enough to make this denial technically true.
Once happened while remarking on the Fruit Loops shake at Burger King. Opened Facebook minutes later and got a Fruit Loops ad. I assure you I don’t message, post, or in any other way remotely suggest an affinity for Fruit Loops. Commenting on it at Burger King was the definition of a one-off incident.
I guess another possible explanation is some very sophisticated combination of location tracking and Kellogg’s targeting people they know will be primed by that promotional item. And then of course it could always just be an amazing coincidence.
All I know is I revoked microphone permission after that.
EDIT: Something just occurred to me. Facebook doesn’t have to be the one using your microphone to target ads. The ad buyers just have to have a way to match your phone to your ID (which they can easily; there was a site posted here a while back that sells that) and then listen in through ANY one of the apps you have installed. If this were the scheme, Facebook could reasonably get away with saying “WE don’t use your microphone to target ads.”
I was once sitting at a bar having a beer, chatting with the bartender (one of my best friends) and the guy sitting next to me, whom I had never met before. I pulled out my phone to show my friend something that had been posted on Facebook and, while browsing Facebook right after that, the app showed the guy sitting right next to me as the first person under "People You May Know". I had not "checked in" or posted any updates while I was there, FWIW. It was at that point that I removed all permissions (i.e. location tracking) that I could from the Facebook app.
More on-topic: I recall reading that the Facebook app would activate the microphone when one was posting a status update, apparently listening for things like a movie playing in the background and such.
After Obama’s comments on the NSA surveillance program back when sh*t was hitting the fan, I’m always keenly parsing these kinds of statements for word play. There’s usually a way out of accountability and liability through evasive rhetoric.
I agree. Really, I don't see why Facebook wouldn't monitor audio like this for targeting ads. If you make money from serving ads, that seems like exactly the kind of thing you would want to do.
Like you, I have gotten pretty good at noticing "weasel words" and such in the government's statements (denials). It is clear that they take their time to make sure they word their statements ever so carefully.
> Remember the time they kept the app open in the background on iOS devices by sending an empty audio buffer to the phone for playback?
> People started discovering that after iOS introduced the battery/power usage section in the Settings app.
They couldn't do it 24/7, of course, but they could certainly take "samples" at regular intervals or in certain locations or something. From my own experience, it doesn't seem that Facebook has ever been worried about battery life or preserving it.
Anecdotally, Facebook battery performance is pretty poor, it's always the app that drains my battery the fastest. I don't think they care about power consumption.
There's a very significant difference between "Facebook's battery impact is pretty shitty because the app is clunky" and "this would mean the average smartphone standby time would be two hours because it would require a phenomenal amount of power."
>Really, I don't see why Facebook wouldn't monitor audio like this for targeting ads.
Because they'd get hit by huge lawsuits and fines from governments all over the world. Europe is especially hungry to fine US companies for privacy issues.
Also it would be terrible for their reputation. Hordes of people would uninstall the app, and even delete their accounts. I would for sure.
It's becoming harder to avoid their apps now that people I need to stay in touch with are on the inevitable Whatsapp that needs voice+other permissions, plus whatever other apps they bring into our lives next. I think it's kinda hopeless.
Facebook may not be recording audio, but they do pay attention to your cell phone, GPS, and wifi, so they know that you're at a particular fast food place. Then the brand-named shake is probably something they're trying to advertise, maybe even time-limited, so the fact that you're at that place is enough. Past that it's coincidence, the law of large numbers, and confirmation bias - you remember that instance and no others, and while a few people have their own anecdotes to share, HN's readership is large enough that FB could produce that population of anecdotes with a hit rate of "One time out of ten, advertise $restaurant and mention a single totally random item selected off their menu".
The "People you may know" anecdote below is, similarly, some combination of location, wifi sniffing, luck, and confirmation bias.
> then listen in through ANY one of the apps you have installed
If there was some third party ad library that was installed across multiple apps that streamed audio and/or performed local speech recognition on everything you said don't you think somebody would have a) noticed by now, b) a security researcher figured it out, or c) everyone's battery's would be running red hot all of the time.
Teams at Apple and Google reverse engineer and debug the top 100 apps on their platforms all the time (there are teams dedicated to doing it). Facebook is also not the NSA - people leak stuff from them all the time. You're suggesting that either everyone is complicit in this, or there's a conspiracy so large and technically complex that it's managed to be kept secret all this time with no technical evidence outside of anecdotes.
Presuming Facebook has some sort of totally encompassing omniscience is a little foolhardy in light of e.g. the Russia issues, isn’t it? I tacitly admitted they may at worst suspect but not know that third parties are targeting in this way.
I’m not sure I trust that security researchers can be relied upon to catch everything. There are so many flagrant abuses of user privacy that go unchecked or remarked upon and nevertheless ignored that such an avenue of prevention seems highly unreliable.
Also you’d be surprised at what people are willing to do when you define a certain culture and incentive structure. While you may dismiss it, I find it frankly astounding that the NSA program managed to metastasize into the behemoth it became before it was finally outed to the public. And even a cursory reading of behavioral psychology will help to explain how that can happen.
I also know that Facebook perversely prioritizes metrics above pretty much all else. And it means they’re perfectly willing to indulge in dark patterns if the numbers line up.
Yep. My immediate read of the denial was, "the legal entity that is Facebook does not itself spy on users in this exact manner".
Once they have a business model that does not revolve around selling data about their users to their customers, claiming they don't spy can be something other than a bad punchline.
Out of curiosity, why did you remark on the Fruit Loops shake? Could it be due to an ad campaign that started prior to your remark, and continues after it?
We were going through the drive-thru and saw a sign. It seemed so magnificently revolting, I wondered at the time if it was a red herring to drive word of mouth.
What about the possibility of geofences detecting your proximity to a Burger King and perpetuating through ad networks to trigger the current ad campaign which happens to be the Fruit Loops shake?
In my case I was talking about flying with some coworkers and mentioned the entertaining safety videos that New Zealand Air had done. Probably hadn't thought or talked about it for 3 years. Opened up Facebook and had an ad announcing their new video. The microphone permission was turned off on Facebook, Messenger, and Instagram on my phone too, so either Android isn't blocking permissions properly, or they are getting it from somewhere else which is even worse.
Why would any rational human being search ‘Fruit Loops shake’ on Facebook? Let’s also assume for the sake of argument that I only associate with rational people.
You think they need microphone access for that? Froot Loops is in the middle of a massive co-branding campaign with Burger King, and you are surprised that you'd get an ad for Froot Loops while at a Burger King? Occam's razor says no.
Many people here may have missed something. From the link in the article, Facebook does listen using your microphone. They say so themselves, they say that you have to give the app permission, and that it's used to listen to say a background music or TV show. they also deny using it to target ads, but they say they do use it for other things.
now, I also remember them writing that they only listen when the app is open and a status is being written, but I'm not sure about that statement. Anyhow, the key thing is that Facebook does overtly and publicly use your microphone with its app.
We can look at the OS permissions to see what it’s doing in the background. Facebook definitely doesn’t have the ability to listen to the mic in the background on iOS unless you’re implying that Apple created a back door for them...
The thing that frustrated me with Lance Armstrong over so many years was the smirk on his face when asked whether he used drugs, and he would reply, "Hey, I passed the test, isn't that good enough for people to stop asking?"
I suspect Facebook is falling in that same boat - technically following rules so they can answer questions the right way. But we're not seeing the smirks on their faces as they give that answer.
Besides Facebook denial and the complete lack of empirical evidence, what are you looking for? Why do people keep talking about "this one time it maybe happened", when it should be happening to everyone all the time, if it's happening at all?
I was totally skeptical when I first heard stories about people's feeds being curated to suit what they were talking about in front of their computers, but then a couple of things made me rethink my views.
I was in the car with my friend once (both our phones were in the car with us too), and we were driving and it started raining. His wiper blades were shoddy and doing a bad job of clearing his windscreen. We had a 5 minute discussion on cleaning blades, where was the best place to get a replacement, etc. Neither of us picked up our phones, or did a search on 'wiper blades', nor asked Siri etc. about nearest places to buy.
But the next day, for the first time ever, I started seeing ads for wiper blades on my feed. They disappeared after a few days, but it was spooky, as it was for an item that I have never ever discussed, or seen on my feed before, or since.
Another anecdote. I was sitting working in my office the other day, when my son came in to talk to me about something. While chatting, he picked up a bass guitar that sits behind my desk and started playing the riff from 'Seinfeld'. I may have said something like "Oh, the Seinfeld theme - you know that was played on a synth and not an actual bass guitar?". Our conversation though, was about something else altogether.
A day later, on my feed, I see an ad about "How they played the Seinfeld theme" [0]. A totally specific ad like that, within 24 hours, totally unrelated to any searched I had done in my past. This one too disappeared within a couple of days.
> But the next day, for the first time ever, I started seeing ads for wiper blades on my feed. They disappeared after a few days, but it was spooky, as it was for an item that I have never ever discussed, or seen on my feed before, or since.
Could be something like this: Facebook knows from location data you were driving/riding together. Soon after your ride, your friend searches for wiper blades. Facebook then shows you wiper blade ads based on that knowledge.
The problems with this 'psychological explanation'
1. It is just as unscientific as saying that Facebook MUST be listening, without finding any evidence in the workings of phones, app source code, etc. Almost by definition, there is no experimental evidence for the Baader-Meinhof Phenomenon.
2. It posits that everyone who has experienced this is unsophisticated and ignorant about cognitive distortion. Clearly, some people who believe that Facebook is listening to them talk also believe that there were no planes on September 11th, etc. However, many many people today have a good understanding of psychology, are well aware of (basic) cognitive biases, have thought about these experiences. People also have experiences of unlikely events but which have well understood or easy-to-estimate probabilities, which didn't occur much historically. Lots of people can distinguish quite well between their own illusory biases and events which are coincidental, but not 'in the mind'. Some of these people seem to also believe their speech has found its way into advertisers' profiles.
> 2. It posits that everyone who has experienced this is unsophisticated and ignorant about cognitive distortion.
Not really, it just suggests people aren't perfect statisticians and perfect at recording information. Which we aren't.
> People also have experiences of unlikely events but which have well understood or easy-to-estimate probabilities
The probability of seeing a relevant advert is not something we can easily estimate.
> Lots of people can distinguish quite well between their own illusory biases and events which are coincidental,
I would be shocked if this was even a reasonable percentage of people. I am certainly not a perfect innate statistician with perfect recall, and I think most other people are not either. Humans are just bad at this stuff.
> which didn't occur much historically.
I'd be shocked at anyone with a very solid recall of all the adverts they see.
Without strong evidence, this seems like a perfectly reasonable base view. It'd be a complex, expensive, power and data hungry way of targeting adverts, extremely controversial and possibly illegal. There's no evidence they're actually doing it, no whistleblowers, and many of their adverts are just terribly targeted.
People here are bringing up examples like a young couple being advertised a pregnancy test. Hardly requires invasive monitoring of a billion people to achieve.
I would disagree with that phenomenon. In direct contrast, my awareness of these ads kicked into overdrive after seeing them for the first time, so I was hyper aware and looking out for them afterwards, which is why I noticed they disappeared a day or two after popping up.
Since then, my feed has been a bland sea of adverts for mattresses, knives etc. that almost all of my friends complain about seeing.
I don't use ad blockers, due to my development work in web apps etc. I like my browsers to be as stock as possible, and I do actually at least glance at nearly all ads I see in FB etc. just to see how their targeting is being skewed by my search history or messaging online.
These were two quite distinct cases of ads that I've never seen before, which popped up within 24 hours of me talking about (but not doing anything else online related to them) and then disappearing soon after.
Sure, an anecdotal sample size of 2 could just be pure coincidence, but something about it still doesn't sit right with me.
It would be weird if that sort of coincidence never happened. Two incidents of this happening to you ever doesn’t sound like much. Maybe even a bit low.
If they really are using audio for ad targeting like they do with search history, then it should be easy enough to prove it by experiment.
Sure. I am used to, nay, I _expect_ ads for software tools and music gear given my work and hobbies in my life, as I frequently talk about those things to my friend on FB and IRL all the time.
But wiper blades is so off the charts for me, that I was totally surprised to see about 3 ads for them the day after I had a discussion in the car. I don't even go to sites remotely associated with the automotive industry. Not even locality based, as it was a full 4 months out of the real 'wet season' we have later in the year. (NOTE: Not even in our wet season did I see random ads for wiper blades).
Having said that, I won't be surprised to see more wiper blade ads now that I have been talking about it on HN - who knows if they have linked up the data from online forums back to FB etc.? The handle I use here I tend to use on nearly all online forums I belong to.
> I don't use ad blockers, due to my development work in web apps etc. I like my browsers to be as stock as possible, and I do actually at least glance at nearly all ads I see in FB etc. just to see how their targeting is being skewed by my search history or messaging online.
You are doing yourself a disservice. Have two users (or at least two browsers), one for work, which you keep ad-blocker free, and one for life, which you shouldn't.
Just a note, not trying to pick your anecdote, but the experience of something seemingly occuring frequently out of nowhere is exactly the baader-meinhof phenomenon. You’d need to demonstrate a causal link to firmly disprove it, which would assumably be difficult in this case.
I think that it's more likely that the companies that manufacturer wiper blades run targeted ads in places where it's raining and that your son looked up the sheet music for the Seinfeld theme on his phone and Facebook thought you would like it because you are Facebook friends or because your GPS coordinates were next to each other.
Fair call - this reasoning I can relate to. I was the first spot of rain in a long dry season spell, so it stands to reason that drivers in the area were highly likely to have dried out, unserviceable wiper blades just when they needed it.
The link about friends search history and close GPS co-ordinates also makes sense.
I had a similar experience in June when I was on a car journey and we got a crack in the windscreen. My phone was out (was using it to navigate) but it wasn't touched at all. It was a Sunday, and I didn't bother researching how to get it repaired.
Later on when I checked my phone I started getting ads in the Facebook feed for windscreen repairs.
It’s useful to specify iOS or Android in the anecdotes. iOS will show a banner at the top of the screen when an app in the background is using the microphone. On Android, the last I’ve seen, this isn’t the case.
> Here’s how it works: if you choose to turn the feature on, when you write a status update, the app converts any sound into an audio fingerprint on your phone. This fingerprint is sent to our servers to try and match it against our database of audio and TV fingerprints. By design, we do not store fingerprints from your device for any amount of time. And in any event, the fingerprints can’t be reversed into the original audio because they don’t contain enough information.
And my previous comment :[2]
> Notice how they say they do not store sound or recordings. But what if it is converted to text?
Notice below they say that the fingerprint cannot be reversed into the original audio because it doesn't contain enough information.
Text cannot be converted to the ORIGINAL audio but it can be converted to audio.
This may be just me being paranoid, but I do know that legalese is designed exactly to say something without saying everything.
Dozens of times in the past few years I've been served relevant ads on some topic instantly after discussing it in person in proximity to a smartphone. I've encountered countless other anecdotal stories from friends. When this happens I feel an acute sense of violation and vulnerability. It's hard to imagine a less sinister explanation for this phenomena.
This seems like something that could be settled via some kind of study, like playing audio to the phone and recording what ads get shown and doing it to a population of phones and social media accounts.
> It's hard to imagine a less sinister explanation for this phenomena.
You just forget all the other signal you've giving. Funnily enough, last week somebody was trying to convince me that Facebook was recording their every word.
"I was telling a friend about a recent trip to Thailand", they said, "and I immediately saw an ad for Thai Tea". I asked if he'd used Facebook when he was in Thailand. He had.
Which is more likely: Facebook is recording every conversation of every user on their app and mini them for ads or the advertiser in question just targeted people who had recently been to Thailand.
You can't imagine a less sinister explanation other than Facebook or some other entity is secretly recording your voice and the voices of millions of people, in violation of numerous state and federal laws, through their smartphone microphones, uploading it to their servers surreptitiously, performing voice analysis on it to determine who is speaking out of 350,000,000 possible cases (presumably they've already cataloged everyone in the country somehow?), and then use that to serve you more targeted advertising?!
True, it seems far-fetched at first. We should keep in mind that a even few short years ago theorizing about the US Government operating a widespread domestic electronic surveillance dragnet was still the realm of delusional conspiracy theorists.
I don't see how that matters. Besides, people have been discussing widespread, domestic electronic surveillance for decades far outside conspiracy fringes. The Clipper chip, as just one example, was a big deal at the time and received widespread, mainstream coverage.
I mean, they did break iOS platform rules and pulled some tricks to keep the app open in the background all of the time by sending an empty audio beffer to the phone for "playback."
Not sure if it is so far fetched to do some lightweight voice analysis (client side) and store significant terms in the same way that you would store search history.
I can add to that anecdote... I was discussing which podcast app was best for Android with a friend, and in the presence of her Android phone. The next day, she started getting served ads for podcast apps all over the place.
Keep in mind, this was someone who would have been completely new to podcasting and have never searched for podcasts or podcast apps before.
I was using google navigation and someone in the back asked "what do you mean one-size fits all"... then my phone which was only running navigator chimed in "one-size fits all means blah blah blah" was definitely creepy.
Google Maps navigation feature listens for verbal input (although it's supposed to be triggered by the wake word) to do things like add stops and such without touching it
No microphone access is needed for the spooky ultra-targeted advertising claims. You just have to keep in mind that Facebook is cross-referencing all your information with everyone physically near you (even people without Facebook accounts, as they can be extrapolated from their relatives and from photo geodata and tags), and cross-referencing all of those results with cookies from ads all over the web and with purchase history results from credit card companies and retailers.
They don't at all deny listening to conversations in this article. They only deny using that information for advertising.
Now about the confirmation bias thing, it seems more likely to me that your credit card history or the store member card purchase history is being mined.
A few weeks ago, I went into a Lenscrafters because I needed some cleaning wipes for my glasses. I have Wifi, Bluetooth, and Location services disabled for Facebook.
Someone messaged me on Facebook Messenger, I responded, put my phone back in my pocket, and finished the transaction, and then left Lenscrafters and walked down to the store that I was actually in the mall to see.
Within the day, I started seeing contextual ads for eyeglasses in my facebook feed, along with ads in other apps for eyeglasses.
I'm only suspecting that there's a service that does some flavor of ultrasonic beacon/listening technology, and facebook (along with DFP and the other large ad markets) are buying that service in troves.
Or like the article said, you just noticed the contextual eyeglass ads because you had just been to lens crafters and it seemed weird to you. Also, by contextual ads, do you mean the eye glass ads were related to context they were shown next to?
The other explanation is Facebook has data sharing agreements with the payment instrument you used for the wipes.
I had been to Lenscrafters, but, there were no location services (no wifi, gps, or bluetooth) turned on...just my cell phone data.
By contextual ads, I mean, "News stream ad/in game ad" -- basically, anything that had been targeted to be sent to my device.
I believe I read something about Facebook using DataLogic info (which tries to tie payment instruments to advertising clients), but, apparently the music supplier for LensCrafters also has "Audio Beacon" technology for their clients to use (http://us.moodmedia.com/social-solutions/), so, it could be one of the two.
Rather than contextual, I think “native ad” is closer to what Facebook’s ads are versus Google’s old system that tried to match ads with the content they were shown next to.
From a technical perspective, it is not very likely that they will listen to people's voice to push advertisements. They needs to listen to you ALL the time, collecting your voice from all kinds of noisy environment, uploading to their server for further analysis. And in order for this service to be profitable, they need to train model to extract the part of your conversation that hints a shopping intention.
This requires infrastructure at a really large scale I don't think even FaceBook can sigh up for that. And if they have a legal department, which I assume they do and a functional one, those ideas would be killed before it moves to implementation.
My expectation is that they could sell advertisers activation sounds on TV ads. The Facebook app would be programmed to listen for a small handful of sounds (like "hey sir"), but which correspond with eg a car ad. Then the advertisers can track consumer reach throughout the process. No speech recognition necessary, it's just another row in a database, recording an ad display.
No evidence that this is what they're doing, though.
I was skeptical until I experienced an ad in my feed which in my estimation could only attribute to data captured by a conversation had hours earlier. Of course there is the possibility of coincidence but I find it hard to believe.
I took advantage of the option to see why I was shown the ad, and was told demographic information, as well as a data partnership with DLX. I would like to know if others who see and ad they suspect was displayed due to microphone captured data have this DLX data supplier source in common.
It did something very similar to me and really made me second guess myself, but there is no other explanation other than it used my microphone to listen to my conversation at work. I felt so paranoid about it and was very weirded out.
My backpack zipper broke when I was walking to work. When I arrived at work I told my coworker I needed to buy a new backpack. She and I talked in her office about what type of backpacks her son liked, did they have enough pockets, are the zippers strong. I never used my phone to google anything about backpacks, never looked anything or searched for a new backpack online. That same afternoon- boom backpack ads in my facebook feed. Totally creepy.
> but there is no other explanation other than it used my microphone to listen to my conversation at work
You'd have not noticed if you saw a backpack advert at some other time. Just how many adverts do you see that aren't standing out? Huge amounts, probably.
This could be quite easily explained by the frequency illusion.
Perhaps your coworker googled about backpacks after the conversation, and facebook knows you spent some time together just before that and put two and two together so to speak.
I'm sure most of the time their algorithms make guesses like that it doesn't result in anything relevant, but you don't notice those ones.
If this cross-referencing of other people's searches is something that they're doing, though, that's concerning too. I don't want my partner to see ads for a gift I'm surprising her with because a facebook algorithm is guessing she might be interested in ads about things I've googled.
IMHO this is irrelevant to what is being discussed here.
> enables devices powered by Snapdragon 800 processors to be “woken up” by a custom voice command (beginning with a custom phrase set by the device OEM like “Hey Snapdragon”, for example)
Because they are always on and have to use minimal power, these chips are used to recognise a very specific term (probably why these companies don't let you customise how you summon the assistant), and cannot do anything like voice transcription, network calls, ... (which would just kill your battery).
Why would audio have to be transcribed locally on battery power? You could do that server-side and/or keep the data stored locally as audio until the device is charging and on WiFi.
Record audio data continuously on disk?
I'm not sure if this is even worth debating, it is a fact that this is not happening (these chips are on most new Android phones, and they do not save audio).
Wait are you trying to say there’s no way to record audio on a smartphone app? And also, why are you calling it “disk”? Smartphones use flash memory for storage.
What minimal power? Phones are being charged at home all the time. Android apps can prevent sleep. Dsp to parse voice is there. There are sdks for all this.
I fully agree with you on that. However, please grant that mobile devices spend a substantial amount of the day plugged in and can draw full power (people mining coins, etc).
interesting. thanks... I wonder if they have something like this for ios devices? seems like they have similar capabilities via the "hey siri" or whatever it is but I am curious if there is a hack to allow it to use that all the time from userland.
One thing I've wondered was that Facebook began showing the recent photos you'd taken at the top of the app, and prompted you to share them. Are they sending every photo you take to their servers to feed it back to you at the top of your stream? I realize they could implement this feature by displaying local photos without sending them to their servers, but I've never seen confirmation that photos remain on your device only until you post. I uninstalled the app awhile back, so maybe they don't do this anymore, but anybody have any insights into how they handle your photos?
I don't play golf or have any interest in it, but I was talking to my brother about him using golf carts on his farm as vehicles. I kept getting golf cart adverts in Facebook for the next month. Maybe it used location data to suggest we might be interested in the same thing, but it was a bit weird.
I used to dismiss these rumors as paranoia/conspiracy theories in a maybe naive application of Occam's razor. But concrete leaks and tech news of recent years made me a believer. I sure believe Facebook is capable of this, nowadays. The only thing that makes me skeptical is that there should be reproduceable cases someone has documented for concrete evidence. All I ever hear is anecdotes.
The fact that there is no hard evidence makes me very skeptical.
Every anecdote is explainable another way. Someone in the comments said they were having a conversation about storage units and saw an ad for storage units immediately after. Even if they had never searched for storage units -- and can we really be sure? That would be easy to forget -- Facebook might just have extrapolated they they would need a storage unit from related searches, like searching for a moving company.
I would be willing to bet this type of algorithmic extrapolation explains every one of the microphone-ad-conspiracy stories.
> The only thing that makes me skeptical is that there should be reproduceable cases someone has documented for concrete evidence.
I'll throw another layer on the conspiracy pile and say they deactivate that behavior for anyone they notice trying to prove it, similar to Uber's Greyball.
A fairly reliable and simple test is this: leave your phone around content for several hours of a different language and see if ads related to that language show up.
It's surprising to me that people here are still debating this. I believe it's been going on since the original iPhone came out after a friend told me a very specific product appeared for her on Facebook immediately after she spoke about it and went to go look it up on her phone.
I don't think Facebook would stop at any measure to get all the data they can from you if they can get away with it and frankly anyone who does is extremely naive. It's right there available to them. There has never been and may never again be a time when companies (particularly Facebook and Google) to obtain as much sensitive data as is available to them from us without our knowledge, without external oversight, and without repercussions due to the lack of legislation.
I am not sure if this is real. However, there are people on YouTube talking about topics they have never typed, never cared about, or never shared on Facebook and sure enough, the ad appears.
It would be great to really inspect the traffic and see if this is really the case. Independent of what Facebook claims.
Either way, I am happy I have deleted it off my phone. I have never been happier each day not reading garbage and actually focusing my time on apps that improve my life and move my life forward.
Technicalities and choice of words are amazing ways to deny what isn't exactly happening here.
For example, why use the word 'recording' when the problem is accessing the microphone at all... Facebook doesn't have to 'record' to stream the audio to their server for audio recognition without 'recording' anything.
Of course all of this is just from the title of the article but the paranoid will pretty much always be able to find wiggle room.
The kind of people who believe these conspiracy theories about Facebook are not the kind of people who believe in scientific methodology. Fuzzy bigfoot syndrome and all.
I think the risk versus reward for Facebook and other companies is very high. All things being equal, I think it's more likely than not they will record things, will listen for words you say, and use that. The risk is enormous if it comes out with proof, but let's say they only listen a small percentage of the time, that kind of AI analysis is priceless.
No, you’re right. There is zero accountability for this kind of stuff. No US government agency or US-based business has ever suffered for it. I defy anyone to prove me wrong.
I doubt it will make a difference to most people. The staunch pro-privacy stance really only exists on tech-savvy forums like HN. Go on reddit sometimes and observe how people will say "so what if Google has all my info, it's useful and free". If Zuckerberg calling his users dumb fucks for trusting him didn't get them off the platform, a "small" privacy violation like this won't either.
And the general sentiment appears to be very far from what you claim. I think it's possible that this issue is going to come to a head this time around. I don't know what the outcome will be. Just saying it's confirmation bias (the likely real explanation?) may not be enough to satisfy people.
Assume it was perfectly true at the time. Is that relevant to the current state of Facebook, with 20k employees, public stock, and a 33-year-old Zuckerberg?
I'm inclined to dismiss it as confirmation bias, but I've had this experience personally - a conversation followed shortly by an extremely specific advertisement on FB shortly thereafter.
As suggested in the comments on Reddit, there's another possible explanation for this -- someone else in the conversation does a search based on what you were talking about, clicks on a result that sends tracking data to Facebook, and then the ads are retargeted to you based on a shared public IP address.
The last time I noticed this, it was a conversation between my wife and I, and I (we) poked around in her search history for that reason. Nothing that I could see, and it was within minutes of the conversation.
All that said, I still lean towards confirmation bias.
I wonder how likely it is that this sort of thing happens without a mic being involved. Consider that Facebook already mines all sorts of Facebook data, as well as data from websites you visit. They much have a pretty good profile of people's interests.
Given that, it's fairly probably that you will be shown ads about things that you are interested in, and it's fairly probably that you'll talk about these topics with friends. Given this and the number of people that use Facebook, it's probably not that unlikely that your experience happens simply by chance fairly frequently.
The odds of something like that happening JUST AFTER you've spoken aloud about it should be small. Too small even for a dozen "random" events, even considering the number of Facebook users.
My partner is convinced this happens, she even tried to run experiments trying to say random things and see if they show up. None of the experiments worked out, but I have to admit some of the coincidences are interesting. But not so much advertising, just any content that ends up in the feed. I'm more on the side of confirmation bias.
As I’m curious, are you using Android or iOS? I can’t prove it, but I find it suspect this happens on iOS, but I know from first hand experience it’s technically possible on Android.
I worry about facebook and all that and the state of privacy in the modern world, but a lot of these stories sound a lot like confirmation bias to be honest.
What I'm getting out of "We don't use your microphone for ads" is that they use your microphone, they just don't use it for ads. Which as many comments have already stated makes sense because you don't really need the recordings to determine a suitable ad when they are already getting all the data they need from just using facebook.
What I think they do is, they record everything using the microphone, and they store it if you are a person of interest. A person of interest could be that maybe you have a over 1000 friends or something, maybe your a ceo of some tech company, maybe you're a lead programmer at intel, maybe you're someone the US government wants to track, maybe your a senators staff member, maybe you work for the Skunk Works at Lockheed, maybe you said a key word/phrase that throws you into the 'person of interest' category like 'electrogravitic propulsion'. When these surveillance companies do this type of stuff, they do it to people of interest, and if they ever need to put the squeeze on someone, or steal important information like get secret information about a stock trade or something, they go to that person of interests microphone records (which probably get transcribed into text by some program) and search for whatever they want to know. This is why this shit should be utterly banned, it controls people, it steals from people, it destroys innovation and scientific progress before they ever get off the ground, and its just plain manipulative.
One thing the internet does which is super incompatible with how people deal with information is that it amplifies anecdotes and selection bias.
If you were just talking to a sample including a small fraction of your friends and acquaintances, and you found that like 10 of them had noticed that Facebook seemed to be showing ads based on their conversations, that would be compelling evidence that something was up. And since for the vast majority of human history, that was the kind of information you had access to, that's the way we tend to process things.
But now you can go online and mention this weird thing that happened to you, and a thousand other people will all see it. Suddenly you have super high odds of a "cold hit", even though the phenomenon was purely random.
And to make matters worse, it's not purely random. Facebook has algorithms that are trying, often successfully, to model your life based on the information you give it. So the chances of "eery" ad targeting are significantly higher than if it were just random.
I think we're going to see this kind of conspiracy theory more and more as time goes on, and we need to figure out cultural tools to counteract this problem.
On the other hand seeing as this must be done client side then someone should be able to do an investigation and see if Facebook sends voice data to their servers.
And not to mention how many engineers that work on Facebook that could be whistle blowers.
But has anyone scientifically validated any of these assumptions? It doesn't seem very hard. Speak, then visit pages with FB ads and take note.
Seems like the internets has taken these serious accusations as some sort of corporate conspiracy but I haven't seen any independent, credible study of how Facebook's apps (confirmed) access to microphone are actually being translated into ads.
If this is happening then it must require some amount of CPU and memory use on the device, which even if hidden from the OS can be detected via power use and heat dissipation. It should be possible to indirectly measure a phone's CPU load (including any hidden processors) by filming it in the infrared and/or by attaching it to power and measuring the wattage consumed from the supply.
If this is happening it must also require some network I/O. Something must be phoning home and reporting the data if the data is to be useful.
Note that there is an intrinsic trade-off between the heat dissipation angle and the bandwidth angle. If the device is doing parsing, speech recognition, AI, etc. locally then it must be consuming more energy and dissipating more heat. If the device does these things remotely it must upload more data. Compressed audio is far more bandwidth intensive than digested meta-data.
This seems like something security researchers can and should look into.
Wouldn't it be easier to reverse engineer their app? It seems easy enough, at least one the Android side. You could use something like baksmali to convert all the dex code to smali, then grep that for calls to AudioRecord/MediaRecorder and figure out what triggers that code.
That's why I'm skeptical that Facebook would do this -- it seems like a terrible business decision given the high chance of getting caught, either by reverse engineering or a whistleblower.
A few days ago I ate a chili pepper out of boredom, and after finding out what a bad idea that was, I opened Reddit. The frontpage of /r/todayilearned had a post about someone learning that birds can't detect capsaicin and that peppers evolved it to deter mammals (which have grinding teeth).
These kinds of cosmic coincidences happen to me and everyone else every few weeks, and they mean absolutely nothing. We're just wired to see patterns. I (and millions more) see hundreds of Reddit posts per day, you (and millions more) see hundreds of ads per day. A crazy coincidence every 10,000 ads isn't very strange.
Something like this should be piss easy to prove, especially on Android. Traffic monitoring, monitoring when exactly the microphone is accessed on a rooted phone, anything really. The fact that this rumor has been going about for years and nobody has found anything, means it's just nonsense.
It is interesting that they feel the need to officially deny it. The Twitter user co-hosts a podcast where users can call in to tell him whether they think Facebook is listening or not.
One could argue, from a PR perspective, it does not matter whether Facebook is listening or not.
First, some users do not trust them. Maybe Facebook is worried about those users telling other users.
Second, some users believe it is technically feasible to listen.
Finally, there is no legal/regulatory framework in the United States to effectively stop Facebook from listening[FN1], if they wanted to do so. They have clear business incentives to listen because the objective is to gather more personal data about users than their competitors.
NB: Added "in the United States" for clarification.
FN1. Incorrect? What are the theories of liability? What are the sources of law or regulation in the United States? Is injury required? What is the injury? How does one prove the injury?
As long as it is possible (closed source software and hardware which is unaudited by trusted organizations), then it's going to continue to be assumed to be happening.
If they do listen and present you ads based on your conversation, can't you fool it (and test it) by talking about (let's say) buying new running shoes? Shouldn't you observe ads about running shoes?
That's correct. Apps must request access to the microphone, and iOS shows an indicator while the app is recording.
This is pure confirmation bias. If someone tests this and doesn't get a relevant ad from Facebook, they won't upload a video or tell their friends. Note that even in this confirming case, it took two days before the guy saw a cat food ad. We don't know how many other ads he scrolled by before that one.
I don't see definitive proof one way or the other, but I've never given Facebook microphone access on my iPhone. And many others users claiming they've received targeted ads haven't either. But something interesting that is being pointed out on Reddit is that they can use your gyroscope to detect vibration and convert it to soundwaves when your phone is on a flat surface. The PM at Facebook denies listening to the microphone, but what about the gyroscope data? Technically, what he is saying could be a true statement, but still invading privacy just as much.
Regardless of whether or not they are doing it purposely, the capability is certainly there. But the other data they have from cross referencing other web traffic, financial data, etc is probably much more powerful than what they could get from listening in to our conversations.
However, I still turn off microphone access for facebook and instagram. It's probably not fool proof, but it can't hurt.
Why don't we just try this out, get a fresh device and block location services then create a new Facebook account. Make sure you don't search for the thing you're aiming to get Ads for (Cat Food, Carpets, Jumper, etc.) on the device itself or nearby devices. Then just talk about those things and see if the ads come up.
No one should trust what they say they can or cannot do. What we can only assume is their capabilities in doing it. This is where end-to-end encryption is important. If you can't encrypt it in a way that only the other end can see it, then assume that someone in the middle can eavesdrop it.
There have been plenty of stories of malware eavesdropping over smartphone microphones in the last two years.
I doubt Facebook does it with their app. It's too much trouble, blatantly illegal and easy to get caught. It's most likely malwAre selling the info to fb, who has no idea how any advertisers get their info.
Couple doesn't own cats. Talked about cat food to test if Facebook is listening. Started getting ads in FB for cat food: https://hooktube.com/watch?v=U0SOxb_Lfps
To add an anecdote to counteract confirmation bias: I heard this rumor months ago and we started talking about motorbikes with smart phones held up for a few days (20+ mentions). We saw no motorbike ads.
I wonder do FB buy user ad preference data from 3rd parties, that may in turn buy them from another listening app.
If I was FB, I'd either process the audio on the device to avoid network traffic, or buy the results from a partner, to keep the actual listening at arms length.
It wouldn't be that hard to imagine another ad-tech company collecting the data in some other app and feeding the data back to FB. I can't imagine FB would be asking as many questions from a 3rd party as it would to internal teams.
Once it happened to me that Facebook has shown me under "People You May Know" a person that was on the same WiFi as I was. We had zero mutual friends and no known connection prior.
According to this post on Reddit, Apple gave Uber access to a private API. He's speculating that Facebook might have access to a private API and is using it to spy on its users. Does anybody have more information about this?
I think people are really underestimating how much they can use your friends search data (especially ones you have been in close proximity to) for your ads. Or how big their interest "niche" is. There are only so many categories of web users, even on the very fringe there are probably 10k who think and search just like you. And when they are searching/clicking on products, Facebook will show you similar things.
A girl I was seeing years ago was looking for a new apartment. She texted me telling me she found a new apartment on 20th and Gladstone Ave and asked me if I wanted to go with her to see it. Minutes later, I get a Google Now update saying "It would take you 43 minutes to get to 2020 Gladstone Ave if you leave now".
I thought it was cool at first - Google Now was getting really helpful with the driving suggestions. I also thought it was strange that it was for a specific address - I thought that was a glitch, or the nearest address to that intersection, although Google Now has worked with intersections in the past.
I thought Google telling me the time it would take to travel to her apartment showing was impressive enough to actually tell this girl what my phone did. She was not amused and thought that I had hacked her phone. She wanted to know how I knew the exact address of the apartment!
I was curious how this could have been possible since she never texted me the exact address and found out she did look it up on Google Maps. So it's clear how Google had the data but sharing it with me seemed like a huge breach of privacy.
Perhaps the craziest part of all this is that not 5 minutes after telling this girl what Google did, when I went back to take a screenshot of the suggestion - the Google Now card CHANGED to 20th and Gladstone Ave!
I was seriously wondering they were analyzing my conversation afterwards expressing my concern of this breach of privacy and had the "privacy invasion" level turned down a degree.
I know that smart TVs and the now defunct Kinect have the ability to monitor conversations. Hell I remember when I first used Soundhound! I was flabbergasted, what will they think of next.
But I also know corporations are usually scrupulous about following laws so unless there is something in Facebook's ToS/EULA that mentions doing this I'm skeptical.
> so unless there is something in Facebook's ToS/EULA that mentions doing this I'm skeptical
I don't see why the vague terms in their data policy [1] couldn't include audio (all-caps are my own emphases):
> We collect information FROM or about the computers, PHONES, or other devices where you install or access our Services, depending on the permissions you’ve granted. [...] Here are some EXAMPLES of the information we collect:
> - Device locations, including specific geographic locations, such as through GPS, Bluetooth, or WiFi signals.
If device location is merely an example of information they gather from my phone then why couldn't audio just as easily be another example?
What can we do to prevent this? Use another service? Make it more difficult to link information together with use of per-website generated emails, burner phones, etc? Just seems there is room for something to come along and change the game, just not too sure what that thing is.
I'm not versed in the facebook tos, but it very well could be included. If that bothers someone they should just stop using facebook, what's so hard about it?
Sooner or later we'll have to get in the offensive and break their business model by rendering ads ineffective. It's either that or the Internet we used to know won't exist for much longer. As long as they don't seem to give a fuck about user privacy we don't need to play nice.
Basically the same discussion on separate stories covering the same topic, both on the front page simultaneously. The second only took off after the first was flagged off the page or something.
I didn't want to summon the dupe police before I had a chance to review it a bit closer. Also didn't want to lose both to the flagging committee.
Mostly planning to review participation by various HN users. #facebook #haterade
PS. Neither one is worth an up-vote from me, let alone adding as a favorite!
Facebook obviously doesn't do this, but I wonder how this will end. Lots of people wanting to regulate Facebook right now, maybe some ineffective laws will get passed.
There's an easy way to verify this: run tcpdump and record it happening. The fact that nobody has done this, instead only offering anecdotes, strongly suggests that these are coincidences or from other forms of targeting.
Regardless, it’s easy enough to dump HTTPS traffic (pinned or otherwise) if you have root access to the client, which in the case of Android is not difficult to get.
> Regardless, it’s easy enough to dump HTTPS traffic (pinned or otherwise) if you have root access to the client, which in the case of Android is not difficult to get.
Hm, well okay, would you mind explaining the easy procedure that lets you capture Facebook traffic then? I have tried lots of different methods (Fiddler, Xposed modules, etc.) with root access and have failed. It's no problem in theory, but practice is another matter...
If you have root access to one end of encryption, you necessarily can inspect and save it. The way I would do this is by issuing a certificate for facebook.com that I mark as trusted on the device. This will let you do a man in the middle "attack." But you can probably do this directly on the device: just look for where the encryption is taking place, and intercept it.
> The way I would do this is by issuing a certificate for facebook.com that I mark as trusted on the device.
Half the entire point is "mark it as trusted" doesn't work when the application has already pinned the certificate it's expecting. Have you actually done this yourself at all?
> But you can probably do this directly on the device: just look for where the encryption is taking place, and intercept it.
"Just" intercept it? You mean "just" spend several weeks if not months trying to disassemble/decompile their code, figure out how to inject your own, somehow locate the relevant in-memory data structures for encryption, & reliably patch them at runtime? all while preventing the application from crashing? That's "easy" to you? Have you done any of these things you're suggesting yourself? How often have you done them? and how long have they taken you that you found them "easy"?
I've done all of these things since it's my job. I haven't tried Facebook, because I don't have any confidence in the surveillance hypothesis, but my first guess being inapplicable doesn't change the fact that root access would allow people to prove this is happening, and that hasn't happened.
Furthermore, this could be proven with some fair reliability using correlation only. Is more encrypted data sent when you're speaking? Is more encrypted data sent when the microphone permission is enabled? Does the app access the microphone while sleeping? Nobody has presented anything _close_ to evidence.
> I've done all of these things since it's my job.
It's weird that this is your day job and yet you tell me that I should mark a certificate as "trusted" when we both explicitly acknowledged that the problem was with certificate pinning. You didn't answer this part: how long does it take you to manage to intercept Android HTTPS traffic for a brand-new, never-before-seen application that uses certificate pinning on your day job?
> I haven't tried Facebook, because I don't have any confidence in the surveillance hypothesis
Well then try it with Facebook. If this kind of thing is really your day job then it shouldn't take long, and you'd do everyone a favor by (a) showing that nothing is going on, and (b) teaching people how to do it themselves so that the myth doesn't keep spreading. People would appreciate it.
> Furthermore, this could be proven with some fair reliability using correlation only.
No, it can't. They don't need to be sending raw audio. They could just do some rudimentary speech recognition and send it along with some other routine data.
> Does the app access the microphone while sleeping? Nobody has presented anything _close_ to evidence.
I've personally logged it accessing the microphone when I've been scrolling on my news feed. Though I don't see why you'd believe me anyway.
Facebook is listening to your data--to all of our data, all at once. They have locations, searches, clicks, messaging, photos, hashtags, and any other form of browsing patterns for everyone in your country, everyone in your neighboorhood, everyone in the same room as you.
They have enough data with such advanced analysis that on occasion they can get really close what you're thinking/doing without you explicitly telling them. They're doing this frequently enough to creep a lot of people out and they're only going to get better at it over time.
What we have here is something like Turing test for privacy: a sufficiently advanced amount of data and analysis will be indistinguishable from surveillance.
It's simply unnecessary to listen with a microphone.
Edit: found this slide from F8 in 2015 where they said they store 300PB and process 10PB (with a P) of data per day https://www.instagram.com/p/0tUjrQKH6R