I just finished reading this great article in Wired (and speaking of Wired, they're putting out some great content - definitely recommend) about Illumina sequencer problems discovered by a researcher. tl;dr The chemical process used by Illumina and other PCR-based sequencers (which make up >90% of sequencing out there) are dirty and the additive chemicals (ala the reagents) used during sequencing can/do contaminate results.
This article re-introduced a topic that I'd let languish for a year or so - that there are some fundamental problems in the sequencing market.
- First problem: chemistry based systems (like all Illumina sequencers) instead of a physical system. By physical system I mean something like a nanopore-based sequencing platform.
- Second problem: The vast majority of sequencing platforms were designed as research instruments, not clinical grade or industrial grade platforms.
- Third problem: Non-deterministic technologies (i.e. you don't always get the same result when you run the same test). Even the bioinformatics software used in this space are not designed to provide deterministic results.
Given those 3 - I suppose at this point I have to say, "Why is anyone surprised that people are getting different results?" We are using variable platforms, with messy processes and then using highly variable and unrepeatable analyses on them.
Over the last 2 years I've gone off and used two different PCR-based sequencing platforms to do analyses on myself. In both cases, the tests were nearly identical, similar hardware, reagents, coverage, bioinformatics pipelines (including doing the analyses myself separately for each to confirm the results received). Simply put: 4 nontrivially different results. Different enough that if I was a late stage cancer patient looking for a drug I would have gotten different results.
My only conclusion - which is the same one I've had for more than a few years now (which you can read in a previous blog post - 'Why Is This Business of Genomics So Hard?' ) is that the vast majority of sequencing technology out there is not ready for prime time - those systems are still struggling for repeatable simple results and certainly not ready for large scale clinical applications. In 2014 I said that I estimated 2018-2020, today I would push it out to 2022 or later (later because the nanopore platforms are not scaling nearly fast enough into large scale usage - partly because of technical issues and lack of investment, and also partly because Illumina's juggernaut razor blade business is blocking competition).
If you are going to jump into this space - realize that you're building on a shaky foundation and act accordingly. As I have said previously, we are in a situation where there are nontrivial problems not only in the technology but also the business models, integrations with other healthcare services and finally the education and training of the people responsible for healthcare delivery. Until all of those see significant improvements, years and decades in the future