Testing is a False Hope

Uncategorized 3 Minutes

The condition of “We will use testing” to decide to relax restrictions is a security blanket of thin cloth.  Basically, we’re screwed by mathematics.  What you are about to receive is a proven fact, i.e., a truth.  If you wish to know more, it’s called Bayes Theorem on Conditional Probabilities.  After you read this, you will know what Trump’s and Northam’s advisors have ALREADY told them.

Because the panacea bonfire of the new “Abbott Labs Antibody Test” is being stoked, I have been forced to revive some brain cells that have been comfortably soaking up rum for the past 8 years just to answer one question:

“How much help is random testing for CoV2 going to be in deciding to ‘Open Up’ Virginia?”

Let’s use Abbott’s EUA application[1] for our Gold Standard for answering this question.

From the test results that Abbott submitted to get the FDA Emergency Use Authorization (EUA), Abbott declares that the 95% confidence interval for the Probability of Detection, Pd, of the virus is (94.0, 100), greater than 94% but less than 100%.  In addition, they tell us that the 95% confidence interval for the Probability of a False Alarm, Pfa, is (0, 11).  See the last or next to last page (alpah =0.05, CI) table.

Pretty impressive, well, impressive enough to secure a EUA so to soak up $Billions of taxpayer money, but is really going to help Northam with decisions to end the quarantine measures for Virginia or the USA?  Remember, we’re making an executive decision that could cost Granny her life.

We need to know that, if I select someone from the population and his test returns a positive result then the person really is sick (immune).  Errata: sick would be determined by the presence of the virus, which is the device covered in the EUA application.  The question of immunity, as yet unanswered, would be suggested by a device that detects antibodies.  The manufacturer applied for an EUA for just such a device on 4-15-2020.  The exact confidence intervals for that device are as yet unpublished, but for the sake of this document, we assume they are the same.

We need what is known as the “conditional probability, P(A|B)”, read Probability of event A given event B has occurred.  In our case, P(sick person | +test), or shortened to P(s|+).  Mathematically, this written as

P(s|+) = P(+|s)P(s) / P(+)

where the RHS of the equation is P(+|s), the probability of getting a positive test (+) from a sick person (that’s just Pd from Abbott) times P(s), the probability the guy REALLY is sick/immune from CoV2, divided by P(+), the probability of getting a positive result at all, either through a good detection or a false alarm, or

P(+) = P(+|s)P(s) +P(+|not s)P(not s).

We’re all set to go except we don’t know P(s), and P(not s) = 1.0-P(s).  But we can estimate them.

The US has 600,000+ reported cases, and estimates for total sickened is 10x that, or 6M, out of a population of 360M so,

P(s) =approx 0.1666… Meh, let’s call it 2%

Let’s grab some values from the CI in Abbott’s application, Pd = 97%, Pfa = 5%, P(not s) = 98% and this gives us

P(s|+) = (0.97×0.02)/(0.97×0.02 + 0.05×0.98) = 0.0194/(0.0194 + 0.049) = 0.2836

P(not sick|+) = 1.0-0.2836 = 0.7164

Do you see the problem?  The test sucks since nearly ¾ of the time a non-immune person will be declared to be immune, given that immunity exists at all.  For Virginia, it’s much, much worse because P(s) <0.01 — the results will driven entirely by the False Alarms, aka “False Positives”

[1] https://www.molecular.abbott/sal/9N77-095_SARS-CoV-2_US_EUA_Amp_PI.pdf

[2] This article is a co-publication appearing on Bacon’s Rebellion authored by me under the pseudonym Dean Wortmier and I attest to its authenticity, Nancy Naive.

[3] this one is better and contains corrections

Unknown's avatar

Published by Nancy_Naive

It is far better to have no heaven than to have heaven and hell; better to have no God than God and Devil. better to rest in eternal sleep than to be an angel and know that the ones you love are suffering eternal pain; better to live a free and loving life -- a life that ends forever at the grave -- than to be an immortal slave.

Published

43 thoughts on “Testing is a False Hope”

    1. That’s NOT even a decision. Just wait outside the ABC store until it’s empty, put on your mask, grab your gun…

      Hell, if I could get weed…

      Liked by 1 person

      Reply

  2. Wow.

    I know f-stops, shutter speeds and ISO. I used to know guide numbers, but technology took care of that.

    I am impressed by the math (I am also impressed by the quarter snatched from my ear by my uncle decades back), but the bottom line is that until we get a vaccine or run out of hydrochloroquine hype, we are kind of screwed.

    Maybe we should let the funded protestors go about their business but insist on a BYOV (bring your own ventilator) policy. Or maybe a certain quantity of gowns, masks, gloves, swabs in a kind of fair exchange system.

    Liked by 5 people

    Reply

    1. It is unlikely that any test doesn’t. Note this is random testing. If I were sick, I’d kill to get the test. PDF=95 is good enough for me.

      Random testing depends on selected victim and the apriori probability that he is/was sick. If this is small then the answer is driven by false alarms.

      Liked by 2 people

      Reply

      1. Yeah, you’re right, given the PDF the numbers are what they are and aren’t likely to change much. I’ll be curious to see the antibody test results…

        Liked by 1 person

        Reply

  4. I can’t judge whether the test is useful or not, but I think you shouldn’t conflate infection with immunity. According to your link, the Abbott test only detects the presence of the virus (infection).

    Apart from that, why do you think Bayes’ theorem applies at all? The presence of the virus is unaffected by any related conditions. It either is or isn’t present.

    Like

    Reply

    1. You’re correct… currently sick only, but Abbot has just announced their “antibody test”. When I find their EUA Application for that test, I’ll post the result.

      The presence of the virus in a human is a probability. Bayes applies.

      These calculations are why most companies ended random drug tests, and why my company told the government we wouldn’t even implement random testing. It took me three weeks of writing and rewriting to finally develop a drug policy the government accepted. It was a very wordy statement that said, “we ain’t gonna do it.”

      Liked by 3 people

      Reply

    2. RE: “The presence of the virus in a human is a probability. Bayes applies.”

      Yes it is a probability, but a simple one, not a conditional one. I can see why drug testing employees could be invalid because you can’t rule out false positives caused by conditional factors, such as prescription drug use or poppy seeds in food, but nothing known can imitate the SARS-COV-2 RNA.

      Like

      Reply

      1. A false positive need not have a reason such as poppy seeds. They are a result of the process. They just happen. Poppy seeds just makes them more likely. But, some of these assay devices will “mistake” one virus for another, see Roche’s device for flu.

        Liked by 3 people

        Reply

      2. RE: “But, some of these assay devices will “mistake” one virus for another…”

        I’m sure that’s true, but I still don’t see how conditional probabilities are an issue here. Your “event A” appears to be a “sick person” and your “event B” appears to be the test itself. That strikes me as a self-referential premise in which the formulas you present would only amplify or exagerate the known error rate.

        Not trying to give you a hard time. Just trying to understand why a test with a reported 95% accuracy must be understood as being only 75% accurate.

        Like

        Reply

    2. He’s probably wondering why you PLAGAIRIZED your post from an article at Bacon’s Rebellion. This is a serious matter. Stealing content is illegal.

      Liked by 1 person

      Reply

        1. Yes, you did. It doesn’t matter if you were the author, Dean Wortmeir, which you claim not to be. As a matter of copyright Bacon’s Rebellion gave the piece fixed form. You had no legal right to reproduce it here without attribution.

          Like

          Reply

        2. @Nancy

          That was quite a little back and forth on Bacon’s…

          I thought I followed the stats pretty well and saw your point, but decided to send it to a friend who loves to get into the nitty gritty of such things for his thoughts. He has a PhD in applied mathematics so I’ll be interested in his thoughts.

          Liked by 1 person

          Reply

      2. Nancy is active @ BR. I’ve seen his posts and comments there when you have posted a loing from there.

        So that brings the question: Can you plagerize yourself?

        Like

        Reply

        1. RE: “So that brings the question: Can you plagerize yourself?”

          That’s not the issue. The issue is the article is BR’s content.

          Like

          Reply

          2. The site takes the copyright from the author? I did not know that.

            If I wrote a letter to the editor to 5 newspapers, do they own the copyright or do I. Or make it an opinion piece, same question.

            Cal Thomas publishes under Tribune News. On Thomas’ own site there will be the copyright symbol followed by Tribune News. Yet the same verbatim column will appear in Townhall with no attribution to Tribune and with a different headline. Just signed Cal Thomas.

            So how is that different from NN posting his own work without attribution to BR.

            PS: Before you accuse someone of plagiarism in capital letters, perhaps a bit more civil engagement would be called for until the facts are known.

            Liked by 1 person

          3. RE: “If I wrote a letter to the editor to 5 newspapers, do they own the copyright or do I.”

            Normally, copyright is an “incident of authorship,” meaning the creator has all rights. When you submit a letter or other content to a newspaper, you are signing away your copyright on at least a temporary or partial basis. After publication, your original copyright reverts to you, but that doesn’t mean you can reproduce the published version. If you do, you would be in violation of the newspaper’s copyright.

            You can submit your orignal content to another publication, but if you don’t tell them the material has been published elsewhere, you will make them quite angry, as you would be inducing them to violate the original publisher’s copyright.

            Like

          4. RE: “So how is that different from NN posting his own work without attribution to BR.”

            It’s different because Thomas is a syndicated author. Those who reproduce his work pay for the priviledge. He displays the copyright notice on his own site to let people know that they need to deal with Tribune News if they want to reproduce his work.

            Like

          5. “ Usually, the author of the creative work is the owner of the copyright. But in the publishing industry, the owner of the copyright may be the publishing company due to an agreement between the author and the publisher.”

            https://www.newmediarights.org/book/how_do_i_find_out_who_owns_copyright_book

            Absent a written agreement between BR and NN, I would say NN can publish his column as he sees fit.

            In any case it was not plagiarism since the author himself wrote and reposted his own work. At best, by a stretch, it would be copyright infringement.

            Liked by 1 person

          6. More sources for your information on copyrights.

            “Copyrights are generally owned by the people who create the works of expression, with some important exceptions:

            Employees in the scope of their work…

            Contractors…

            If the creator has sold the entire copyright, the purchasing business or person becomes the copyright owner.

            https://fairuse.stanford.edu/overview/faqs/copyright-ownership/

            You stipultated:

            “When you submit a letter or other content to a newspaper, you are signing away your copyright on at least a temporary or partial basis. After publication, your original copyright reverts to you, but that doesn’t mean you can reproduce the published version. If you do, you would be in violation of the newspaper’s copyright.”

            You are saying that even after publication, the piece now has two copyright owners, you and the paper?
            A partnership? I don’t think so.

            Liked by 1 person

          7. Well, lemme answer, maybe. I have some dozen articles in refereed journals. You submit a paper. They send you a letter telling you it is being sent for review to 3 to 5 persons. You receive the critiques, with suggested edits, an questions. You resubmit with changes or separate answers to the challenges. The journal discusses with the referees.

            Eventually you would receive galley proofs to make final changes, literally with a blue pencil. When you sent the proofs back, you signed an agreement not to publish anywhere else, the “same or essentially the same” work. You also agreed to limit copies at request to the 20 reprints supplied to you. There was a period of time, a couple of years, I can’t remember.

            You sent it in, and the journal set the publishing date, volume number etc. This was for JOSA and IEEE.

            Did that answer your question?

            Liked by 1 person

          8. RE: “You are saying that even after publication, the piece now has two copyright owners, you and the paper?”

            In a sense, yes. In the business, the newspaper typically takes the North American Serial rights, meaning only it can publish and republish the original you gave them for that purpose. You are free to submit the work elsewhere, but you should tell subsequent outlets it has already been published so they don’t assume they are getting the North American Serial rights you already gave away. Many publishers will reject a submission because they won’t get the rights they want, so, you need to be honest and not mislead them.

            This is the same basic practice Nancy describes with professional journals, but there doesn’t have to be any form of written agreement or money paid. Publication itself is the contractual “consideration” and payment in copies is typical.

            I imagine the professional journals Nancy has dealt with would also take a dim view of a simultaneous submission, and wouldn’t bother assigning reviewers unless they were confident they were the only publisher handling the work.

            Like

          9. North American Serial rights.

            I think you are confusing purchased rights and eventual resale. If a publisher wanted to secure certain rights, we need a written contract with compensation. That’s how it works for photos.

            I often sold usage rights limited to specific clients and specific PC ads with time limits with regards to my photos. In one case, the client had changed ad agency and they called a couple of years later and wanted by rights to ads in trade journals. I sold those too. All because I had the copyright as originator of the creative work.

            BR may request a courtesy mention, but unless there was a written agreement, nothing illegal happened.

            Certainly not “PLAGIARISM”.

            Like

          10. Let me add this, you cannot patent or copyright mathematical equations or formulas as such.

            Like

  6. It has been brought to my attention that the body of this post is largely a cut and paste from “Bacon’s Rebellion” lacking proper attribution. Please edit your post to include the source credit or I will have to remove the post for copyright violations, which WordPress does not allow.

    Like

    Reply

        1. But if NN published it on a blog, doesn’t that still leave the copyright with NN.

          Whenever I make a post on here, the button says publish. Is that me publishing or you?

          Liked by 1 person

          Reply

          1. This whole copyright kerfuffle supports the so-called Libertarian unicorn, there would be no problem with it. But seeing as it appears to put a lie to their narrative, instead of refuting it, they jump up and down and say “you stole it”. Classic deflection techniques of the Trumpists in our midst.

            Many times I have seen posts that have “Link to source”, without attributing the source unless you click on it. Is that not similar? Forcing us to click on the link to see where it comes from? Not that I don’t trust the folks here, but who knows what kind of malware is located within some of those links? Just a thought.

            Liked by 1 person

          2. RE: “Many times I have seen posts that have ‘Link to source’, without attributing the source unless you click on it. Is that not similar?”

            No. Linking to content elsewhere is not the same as reproducing it.

            Like

        3. Actually, it appeared first here. It used msword, copied into an email, and sent to James. While I had the copy on the clipboard, I posted here, noticed a few mistakes, corrected them and…

          I then sent a new copy to James, but by that time, he had published the first I had sent him. If I had editing rights there, I would redo both, but I’ve moved on…

          Liked by 2 people

          Reply

          1. You should revise your post here to credit BR’s publication and clarify that you cross-posted the article. That’s bad form, but not the end of the world.

            Like

          2. “Bad form”? It’s his article. He can do with it what he wants. He can post it anywhere he desires. Cross posting between 2 different blog sites has nothing to do with copyright. For crying out loud, he uses the same screen name at both sites.

            And I stand by the thought that if it were a post that supported your’s and Don’s viewpoint, nothing would have been said.

            Like

          4. @Roberts

            You know what is actually “bad form?”

            It is to viciously accuse somebody of plagiarism based on a half-baked understanding of copyright law and ZERO understanding of what plagiarism means (“the practice of taking someone else’s work or ideas and passing them off as one’s own”)

            Here is an example of that sort of churlish “bad form.”

            “. . . you PLAGAIRIZED your post from an article at Bacon’s Rebellion. This is a serious matter. Stealing content is illegal.”

            Like

Leave a comment