Testing Genetic Test Chips
Kevin Kelly
Ann Turner, co-author of the best book on DNA-based genealogy: Trace Your Roots With DNA, wrote me to say that she too has been comparing results from the two big genetic test companies, 23andMe and deCode. She wrote in response to my earlier posting comparing results between the two vendors.
The big news is that places where errors are showing up are probably not random. Here's the argument, starting with her post on ancestry.com:
The two companies overlap on 562,532 SNPs. They agreed on 560,128 calls, or 99.6%. 23andMe didn't make a call on 1,970 SNPs where deCODEme did, and deCODEme didn't make a call on 399 records where 23andMe did. That leaves a mere 35 records where they actually made different calls [see the list below]. In all of those cases, one company would make a homozygous call while the other company made a heterozygous call -- there were no cases where they made a completely discordant call.
Here's the kicker from Ann's letter to me:
Four of those (rs11149566, rs4458717, rs4660646, and rs 754499) were also found in Antonio's list. That's more than you would expect by chance.
Four out of 23 from Antonio's list and four out of 35 on Turner's list of discordant results indicates that these regions (at least) are unreliable.
This is why sharing results is so valuable and a key to great quantified self understanding.
This is a micrograph of the bead array on which these tests are conducted.
Turner's 35 SNPs with different results, if case you also have done a comparison.
rs10435795 rs1045363 rs10743414 rs10945383 rs11149566 rs11179382 rs11707159 rs11915402 rs1209171 rs1221986 rs12907462 rs1303912 rs13422439 rs161381 rs17328647 rs1961196 rs1966357 rs2016461 rs2064034 rs2290516 rs2853981 rs3952469 rs4336661 rs4423481 rs4458717 rs4572718 rs4660646 rs6531490 rs6942478 rs7102702 rs754499 rs7812884 rs845217 rs9332128 rs9476380
How Accurate Are Personal Genome Tests?
Kevin Kelly
I've had my DNA sequenced by 2 of the 3 companies now offering this service to the paying public. I purchased the tests for 23andMe and Iceland-based deCode. I am still plodding my way through the results -- it's sort of an education. One question I had was how well do the two results matched? I give the same DNA to both companies; the results ideally should be identical. DeCode claimed to test for 1,000,000 SNPs and 23andMe for 500,000, so the problem of lining all these results up to see what differs is not trivial. Luckily another user has just done this.
Antonio Oliveira also used both 23andme and deCode. He writes in his new blog:
In order to determine the accuracy of the genome profile provided by 23andMe and deCODEme I arranged to be genotyped by both companies and wrote a computer program to compare the results. The downloaded files contains 576,105 snips in the case of 23andMe and 1,013,349 snips for deCODE. After removing the no-calls and matching the two files by SNP identification, 560,299 snips were present in both files. The comparisson revealed 23 cases in which the results do not agree.
Oliveira made a chart of his results, categorized by chromosome.
The 23 errors makes the agreement between the two sets of data about 99.995% accurate, or an error rate of .005%, which is pretty good for medicine. A better test might be to repeat the test on the same DNA, but I assume the manufactures of the chip have done that. The 23 "unequal" SNPs caught here in disagreement are not SNPs currently associated with any diseases, so these particular errors are inconsequential. I don't know if there are location biases in the errors, but presumably errors can appear in significant locations -- at that very low rate. However if your computer had the same error rate, you'd notice.
"Productivity" Dashboard Monitor
Kevin Kelly
In the annals of self-monitoring tools, here is one that monitors your computer time. It's a fancy version of time management software. You assign certain tags for various functions and websites -- say "surfing" for Digg, Reddit, or Popurls, or "research" for Wikipedia. After you label your activities once, then RescueTime will gather the stats and present you with your accumulative totals in a kind of productivity dashboard. You can get a time budget showing how you actually use time on your computer.
Assuming you can accurately classify which activities are productive, then you can measure your productivity -- at least in terms of how much time you spend on "productive" tasks. (This particular software will also require a certain level of trust since your self-monitoring activities are transmitted to the software's website.) I haven't used it, though it's if free and available on Windows or Mac.
The BodyBugg
Gary Wolf
I'm fascinated by the BodyBugg. Not convinced, but fascinated. This is the most complete self-monitoring system I've yet seen. With an accelerometer, a skin-temperature sensor, a sensor to measure the electrical conductivity of the skin (known as GSR, for Galvanic Skin Response), and a sensor to measure "heat flux" (the rate of heat transfer from the skin), the BodyBugg truly aspires to track a complex behavior – physical exercise – not in terms of outward factors, such as miles run or laps swum, but in terms of inward factors: how much energy has your body used?
This is a hard task, and it's inspiring that somebody has come so far in figuring it out. The goal is round-the-clock self-surveillance:
We recommend wearing the armband as much as possible during waking hours. the more you wear your bodybugg™, the more accurate and effective you will be at maintaining your calorie deficit goal. During low activity level periods of time (such as sleeping), the program will estimate your calorie burn at rest, based on your body parameters, so it is not 100% necessary to wear to sleep.
There is some science available for those who want to calculate energy expenditure through measurements like heat flux. Still, the assumption behind the current version of the BodyBugg is not that users want to experiment on themselves, or participate in scientific research. Instead, they want the Body Bugg to help them lose weight. The problem is that a device that involves such total commitment to rational self-analysis seems ill-suited to such a straightforward goal. If the goal is simply to lose weight, you don't need to measure yourself 24 hours a day, seven days a week. You simply need to eat a little less and exercise a little more than usual. You can track these variables with any calendar program and a scale.
Something like the Body Bugg could clearly do more interesting work. It could show energy expenditure through time, and allow analysis of the relationship between work, sleep, or mood, on the one hand, and patterns of energy use, on the other. It could be used by two or more people, and allow us to test theories of how we influence each other. It could do a lot of fun things. Right now, the Body Bugg is just the technical part of a program of weight loss coaching. But it, or something like it, has a higher destiny.
For people interested in BodyBugg as it is currently intended to be used, there's a good conversation about various issues here. It has apparently been promoted on the TV show, The Biggest Loser, which I've never seen.
Emotion Map of San Francisco
Gary Wolf
How do you feel in different places? The precise correlation of location and emotional arousal is the topic of Christan Nold's long running biomapping project. The project used a simple galvanic skin response meter, which gives a reading of how excited you are.
A GSR device is simple. Here's the Lego version.
These GSR readings are not very specific. They do not tell you whether you are disgusted, shocked, thrilled, or fascinated. But once Nold added GPS tracking, and invited people to annotate their readings, he could produce a map that correlates emotion with locations. This can be mashed up in Google Earth with contributions from others.
Nold's device looks like this.
You can download a printable version of the San Francisco map (PDF). But, better yet, you can get the raw data (kmz) and load it onto Google Earth to browse. Right now this is an art project, a vision of the future, a hint of the utopian upside in surveillance and tracking.
Next step – getting my own version!
Reality Mining at MIT
Gary Wolf
Earlier this week I had a chance to drop in on Nathan Eagle's presentation at ETech about using the Bluetooth feature on mobile phones to keep track, not only of where people are, but who happens to be nearby. This research is part of the larger Human Dynamics Group at MIT run by Sandy Pentland.
Eagle gave a great talk, which led me to read the description of his research at the Reality Mining site. Here one statement that jumped out:
[O]ur ultimate goal is to create a predictive classifier that can learn aspects of a user's life better than a human observer (including the actual user)...
Can our devices know us better than we know ourselves? It seems obvious that this must be true. Human self knowledge is plagued by all kinds of limits: bias, sampling error, memory failure, and lack of sufficient processing power to recognize complex patterns. Machines do not suffer from the first three of limits, and the last is under steady assault from Moore's law. But for computers to help us know ourselves better, they need two things: better data, and new analytical tools for transforming this data into predictions. These are problems that the Reality Mining researchers (among others) are trying to tackle.
In the experiment he described at ETech, Eagle's group gave 100 MIT students free use of a Nokia smart phone in exchange for being tracked whenever the phone was turned on. Some filled out questionnaires, others kept diaries.
In return for the use of the Nokia 6600 phones, students have been asked to fill out web-based surveys regarding their social activities and the people they interact with throughout the day. Comparison of the logs with survey data has given us insight into our dataset's ability to accurately map social network dynamics....Additionally, a subset of subjects kept detailed activity diaries over several months. Comparisons revealed no systematic errors with respect to proximity and location, except for omissions due to the phone being turned off.
Proving that people can be effectively tracked using low-power Bluetooth transmissions has a certain technical interest, but of course the true power of this work lies in beginning to understand what kinds of things can be learned from such tracking. Eagle and his colleagues, for instance, found it easy to predict when two people were likely to encounter each other, as long as the users had fairly regular habits:
In contrast to previous work that requires access to calendar applications for automatic scheduling [Roth and Unger (2000)], we can generate inferences about whether a person will be seen within the hour, given the user's current context, with accuracies of up to 90% for 'low entropy' subjects.
By 'low entropy,' the researchers mean 'easily predictable.' Their claim is that their system can predict social behavior among people who are easily predictable. Such a result might seem the very definition of trivial, but it's not as pointless as it sounds. Such a result functions as a kind of system tuning, a check on whether the basic parameters of Bluetooth tracking and social predictions are plausible. Once you know that it works on the easy cases, you can start trying to generate the more interesting analytical tools necessary to get more surprising results.
Research is being pursued to develop a new infrastructure of devices that while not only aware of each other, are also infused with a sense of social curiosity. Work is ongoing to create devices that attempt to figure out what is being said, infer the type relationship between the two people, and even suggest additional subjects to discuss. These devices see what the user sees, hear what the user hears, and are beginning to learn patterns in people's behavior. This enables them to make inferences regarding whom the users knows, whom the user likes, and even what the user may do next. Although a significant amount of sensors and machine perception are required, it will only be a matter of a few years before this functionality will be realized on standard mobile phones.
To perform these experiments, more than 100 subjects on the MIT campus will be needed. That's where you come in:
While Symbian Series 60 phones have become a standard for Nokia's high-end handsets, they represent a small fraction of today's Bluetooth devices. We are in the final stages of developing a MIDP (Java) version of the BlueAware application that will run on a wider range of mobile phones. The final test of Serendipity will be its public launch on www.mobule.net. We hope that not only will the application prove to be robust, but also quite popular within the realms described above, as well as those unanticipated.
The Mobule site does not seem to be functional yet, though there is a light description of the next phase of the project here, where it is described as a social introduction service. If your phone knows who is in your proximity, it can match profiles and make introductions. To me, this application seems boring and redundant. The world has gone crazy for social networking, but I don't want new ways to make social and business contacts. There is a lot of fear that social tracking will simply be a new channel for exploitative marketing, oppressive government tracking, and annoying, spam-like requests for "friendship." In some ways, the Reality Mining group is underselling their own interesting discoveries, because the promise of new understanding our social behavior goes beyond this impoverished definition of "networking."
Another section of the site offers a clue as to the more interesting applications:
In collaboration with Push Singh and Bo Morgan, we have created an interactive, automatically generated diary application which will allow users not only to query their own life (ie: "When was the last time I had lunch with Mike? Where were we? Who else was there? What did I do next?") but also (after a few months of training data) visualize the model's predictions about upcoming behavior in the immediate future.
The reference to Push Singh and Bo Morgan offers a clue that this work goes deeper than finding friends or hustling sales. The question "what did I do next?" is easily transformed into a prediction about "what will I do next?" Or how about "what should I do next?" The day when we consult devices for advice is closer than we think. It already works in the stock market, and in many expert systems. Many of our decisions are less complex; but until now, both data and models have been missing. Eagle's work is part of a bunch of efforts that will help fill the gap.
In his talk, he spoke of getting the next phase of his experiment going with 100,000 users.
Starting the Life Log Early - The LENA Baby Monitor
Gary Wolf
The New York Times magazine published a story last weekend about a special kind of baby monitor, the LENA, a $400 device that is tucked into a child's clothing and evaluates the "language environment" throughout the day:
A voice recorder tucked into a child's clothing records all the sounds in the environment. At the end of each day, special software evaluates both the amount of exposure the child has had to verbal stimulation as well as the child's own utterances. Ultimately, the device generates percentile rankings that help assess a child's language development, just as doctors provide such rankings for a child's height, weight and head circumference.
The value of the LENA obviously depends on the quality of the analysis. Speech recognition software has come a long way. Security agencies have long lusted after it. But whatever tricks the CIA might be deploying, at the consumer level our devices are not yet able to efficiently recognize and transcribe every utterance, especially in natural environments, where there is high background noise and multiple speakers. In fact, we're not even close. So the inventors the LENA took a shortcut. As Yudhiji Bhattacharjee reports:
The best solution, it seemed, was to eschew the identification of particular words and focus on a recording's acoustic features. Modeling every conceivable sound in a household, they designed a system that distinguishes different voices from one another, gives a rough count of the number of words directed at a child and counts also the number of conversational "turns" that are taken as child and interlocutor exchange words.
The use of sound "signatures" to model behavior is allowing rapid progress in life-monitoring systems. This is the same general tactic used by the inventors of the e-watch, who can track and identify almost any common activity using just three measurements with small sensors that fit on a strap on the wrist: ambient sound; ambient light; and motion, as measured by a small accelerometer.
Technovelgy.com, which tracks the emergence of science-fiction ideas into real life, has a very good post that connects the LENA to other life-logging phenomenon, such as the SenseCam. The SenseCam has long been criticized as a solution in search of a problem. And so the notion of a ubiquitous baby monitor might seem, at first, more a symptom of neurosis than a useful tool. But both these types of systems will eventually prove their worth, and the first stage of this will be in palliating chronic conditions or aiding in the diagnosis of subtle illnesses and disabilities. A story late last year in the MIT technology review described the proven value of the SensCam in helping people with dementia.
There are set of technical papers on the LENA site that explain the system, and compare its effectiveness at analyzing a child's verbal environment with the effectiveness of human observer-listeners. The results are impressive. These are not refereed scientific papers. They are the detailed technical claims of the inventors. But, on the reasonable assumption of good faith, this report (PDF) shows that computer observation is capable of subtle and effective analysis of natural environments.
Wiki Your Genes
Kevin Kelly
I am taking a crash course in genetic literacy by having some of my genes sequenced by the two major genetic sequencing services, 23andMe and deCode. I am still in the process of comparing the two sets of results to see which vendor is better, but while coming up to speed in this new realm, David Ewing Duncan, another self-experimenter, turned me onto a very cool site: SNPedia.
SNPedia is a wiki for personal genomic raw data, which come in SNPs, or in my usage, snips.
Snips are the current desired unit in personal genomics, like pixels in digital photography. For now, more snips is better. A snip is a particular part of the gene that researchers have noticed will vary between individuals (most of the gene does not vary). These single-nucleotide polymorphism (SNP) variations are the tiny spots on your chromosome that are actually sequenced and reported back to individuals. Each snip position has a unique number, and some of these snips such as rs1815739 (good sprinter) or rs795174 (green eye color) indicate particular traits.
On SNPedia you can paste in a snip number -- from the results of your DNA sequencing -- and find out what is currently known about it. Or conversely you can enter a trait or disease and see if there is a snip tagged to it. As information is gleaned from medical journals, wikians add it to the SNPedia. Usually very quickly.
A typical entry will look like this:
rs7495174 is located in intron 1 of the OCA2 gene. The (A) allele (in dbSNP orientation) is associated with blue or green eye color in Caucasians. [PMID 17236130].
This SNP is 1 of 3 SNPs defining a haplotype that has been studied for association with eye color. The full details on the correspondence between the haplotype and eye color can be found on the OCA2 page.
In theory this is what the news personal genomics sites of 23andMe and deCode are supposed to be doing. Only with slick user-friendly designs. They do offer this information, but in their effort to filter this large deluge of data, they both are selecting certain snips as being more important/interesting, and hiding the rest in the page pages of their sites. To surf the ocean of data beyond these selected traits, diseases, or snips is cumbersome. And of course, it costs $1,000 to enter the door -- the price of getting your DNA sequenced at either place.
Here is how I see the nascent field of personal genomic testing shaping up. The retailers are 23andMe and deCode. They don't sequence genes. They outsource that specialized job to microarray manufacturers, while the retailers sell the website interface, and supporting information to consumers. There are only two main manufacturers of the large scale mircoarray chip which is used to provide up to 1 million SNIPS. One is Affymetrix, and the other in the Illumina. Affymetrix and Illumina will sell their microarrays to anyone, although they currently don't sell to individuals. Affymetrix lists the cost of a 500,000 SNIP array chip at $250 today. You need their specialized machines and software to read the chip.
If a third-party vendor were to start selling the naked chip's data for a small fee above its costs, it would be possible to do a large personal sequence using one of these tests and managing your data using open-source wiki technology like SNPedia. Hard-core recreational genomist could probably do a better job than either 23andMe or Decode are doing right now.
This is close to a DIY kit for geneboys. With some mashing of websites, you could get more info in, faster, more personalized to you. If you are already doing this, write me.
Self-Tracking One Hour in Front of TV
Kevin Kelly
Inspired by a French sociologist from the last century, a fellow tracked his family's movements in their TV room for hone hour. He turned his pattern of their locations into a striking info-graphic of the result which he posted on his Flickr account.
He says:
I got the idea from a French Sociologist [de Lauwe] who mapped the movements of one of his students for a whole year in Paris in the 50's. He was shocked to see the narrowness of her existence and the 3x key points that she kept returning to.
Moon River explains:
This is a map by de Lauwe of all the movements made during one year by a student living in the 16th Arrondissement of Paris. Her itinerary forms a small triangle with no significant deviations, the three apexes of which are the School of Political Sciences, her residence and that of her piano teacher, illustrating, according to de Lauwe, the narrowness of the real Paris in which each individual lives and which, according to Debord, ought to provoke outrage at the fact that anyone's life can be so pathetically limited.
Back to the TV room. The contemporary surveyor, Bumblebee, conducting the monitoring process by manual labor. He says:
I used a marked-out equally-spaced grid in masking tape and filmed them moving via video across the grid for an hour. I then reviewed the video and plotted their movements on each minute of the video's timecode onto a 'room map' with corresponsing grid.
The cat's story is one of moving from heat source to heat source and then food. It starts at the heat source -radiator- behind the armchair and then moves over (right) towards the french window where the sun's shining through. It then moves off towards it's food outside of the room - for the diagram's sake this shows it lingering by the door (contrary to the way it looks I didn't lock them in) :-) I liked the little underlying micro narratives that you could take from the map - much like Denis Brown's 'Pumpkin Map'.
Heart Monitors and the Limit of Self-Knowledge
Gary Wolf
The heart rate is among the earliest biometrics used by humans to take stock of themselves. Before mechanical clocks were invented, this was hard. The first doctor credited with making objective measurements of the pulse was an Alexandrian physician named Herophilos, from the 3rd century, B.C., who used a water clock as his chronometer. Using a specified outflow of water to set the time interval, he counted the heart beats of four healthy individuals of different ages, which gave him a base rate against which to compare the pulse of his patients. Genius!
It’s easier to find our heart rate now. In fact, it’s so easy it’s become complicated again. What you used to be able to do with two fingers and a second hand now requires a cardiac monitor, sometimes with a chest strap and a wireless connection to the wearable computer on your wrist. These complications are associated with bigger benefits; we can correlate our heart rate with our exercise regime, for instance. There are more than 700 heart rate monitors listed on Amazon.
Many of the best monitors require a chest strap. Why not just put this in our clothing? The one pictured below is from Numetrex.
Pacing exercise is just one of the things we might want to do with data about the rhythm of our heart. The pattern of the heart beat is a clue to health, and, ideally, it would be tracked all the time. For monitoring serious conditions, how about if we move the monitor from our wrist or our clothing, and put it inside our body? The image below is of a tiny cardiac monitor the size of a small memory stick. It is implanted in a patient’s chest, and recorded measurements can be picked up from the outside.
As we generate more data, the patterns become too complex for our brains to recognize. ECG measurements have to be read by trained physicians. Or by artificial intelligences. While reporting an upcoming story in Wired about the great inventor Ray Kurzweil, who is best known for his reading machine and his theory of the singularity, I found how that his company has also been involved in researching the use of artificial intelligence for the interpretation of ECG. His friend Martine Rothblatt, the founder of United Therapeutics hired Kurzweil’s company to contribute some improvements to the algorithm underlying CardioPal, a 24/7 cardiac monitoring system designed to provide early warning of arrhythmias. CardioPal is produced by Medicomp, a United Therapeutics subsidiary. The underlying algorithm is named Diogenes.
There is a lot of interesting science behind the interpretation of ECG, and it is easy to imagine a not-too-distant era when internal cardiac monitors are a normal health maintenance device, automatically warning of impending problems. The curious thing about this vision of a totally monitored future is that the algorithms that interpret data from these monitors inevitably becomes more and more complex, easily outstripping our capacity for unaided interpretation. We will get a warning of impending doom, but not fully understand why this warning is issued. We will gain more power over ourselves, but not more self-understanding. Maybe we have to adjust our idea of who we are. The artificial intelligence upon which we rely - can this be understood as part of our self?