about being liberal or conservative anymore y'all. That is a hype offered by the fascist whores who want to confuse the people with lies while they turn this country into an aristocratic police state. Some people will say anything to attain power and money. There is no such thing as the Liberal Media, but the Corporate media is very real.
Bayesian analysis of Using Two Independent Surveillance techniques
How much oversight is necessary when using surveillance techniques on a very rare event? How
frequently would innocent persons get caught up in surveillance schemes that are by their nature
upon the judgement of imperfect human beings?
These are questions I asked myself recently, and after a few days I feel that this can be a Bayesian
probability model. In this model there are three events with mutually exclusive results.
You are either "Bad" (criminal, terrorist, predator) or you are not bad .... P(Bad) or P(nBad)
You get a suspect hit with technique number One, or you don't .... P(S1) or P(nS1)
You get a suspect hit with technique number Two, or you don't .... P(S2) or P(nS2)
The P(stuff) means the probability of , or percent of expected occurrences, or rate per 100 persons.
So P(Bad) means the percentage of Bad, or how many out of 100 are gonna be bad.
Since being "Bad" is exceptionally rare, I used an initial rate of one person per 100,000 being a
very dangerous criminal, terrorist, or deadly predator. I also assumed that the potential of
innocent suspects was 1/10 th the rate of actual suspects, or 1 out of 10 potential suspects, which
I agree is somewhat unrealistically small, but for the initial run of the Bayesian test, I assumed a
good faith "best case scenario".
I also assumed the error rates of the two surveillance techniques are decorrelated, or independent
of each other, which ignores the affect of "group think" and time investment in certain suspects
that can often blind side investigations.
The effectiveness rate is the percentage of actual suspects given that you have a positive result
from either surveillance technique. I assumed this was 25% at first.
So what is the probability that you have a non-bad individual as a suspect given that you have a
suspect hit with either surveillance technique number one or number two.
Here is the formula
With the initial assumptions, the percentage of false positives is
=(10^5-1)/(10^5+0.25*10^6-1) = 28.57%
If we assume the effectiveness rate of either surveillance technique is 99% .. . than the number of
false positives is =(10^5-1)/(10^5+0.99*10^6-1) = 9.17%.
If we now assume that the base rate of "bad" is one per 10,000 ... there is no difference. Even if
we assume the base rate of "bad" is one per 1,000 ... again no significant difference.
So, the best case scenario is that 9 out of 100 persons considered a suspect by these two
surveillance methods will be innocent. The only way to reduce this rate is to reduce the number of
false positives. If this is reduced to 1 per 100 the percentage is 1/10 th .. or 0.917%
This is kind of like trying to find something very, very specific with only 10 words ( 1 out of 10
rate) or with 100 words. How many different factors or distinguishing characteristics are necessary
to obtain highly probable knowledge that a suspect is a criminal? It certainly can't be more than
100, or at least that's what I am assuming in this analysis. Very often, in criminal investigations
there isn't more than 10 degrees of information relative to the case at hand.
Keep in mind that you lose accuracy the more characteristics that are included. This is because
each multiple characteristic grouping is a subset of a larger grouping of less characteristics. For
instance, being a male is a larger group than a male with blond hair, or a male with blond hair who
also works night shifts at a bar. The percentage or rate per 100 DECREASES with the more
characteristics you include.
So, for instance, say a suspect has 7 characteristics : went on trip, makes regular large income
transfers, was born abroad, has history of making trips, associates with other suspects, known to
drink alcohol. Not all of these filters are close to 100% predicative, and some might increase or
based upon other characteristics. Even assuming all seven are 99% accurate and also independent
the effectiveness rate will be 93 percent. If one of the factors is only 90% predictive however,
the overall effectiveness drops to 85%. If two of the factors are only 90% predictive, the overall
effectiveness drops to 77%.
This is why I thought 1 out of 10 (90% effective) was unrealistic.
Using these "filters" an investigation swoops up 2,000 persons in a typical dragnet to protect
America, 180 innocent
persons are going to have to deal with legal costs and possible life-long repercussions by
inadvertently getting on a government suspect list.
If we assume 1 one per one hundred innocent rate, this becomes 18 innocent persons per 2,000
And these are both very low-end, best-case scenario assumptions. In the real-world there is a very
large potential to be much larger than 200 innocents per 2,000 suspect list.
This is why you need multiple layers of oversight and very top-level restricted access to these
types of programs. Keeping the collected data secure from malfeasance is another matter.
The whole fabric of the constitution is based upon the credo of "innocent before proven guilty". It
appears to me that the current elites who push the surveillance state as "necessary" and "nothing to
worry about" are turning this national credo around, so that now it seems that assuming guilt is
being used in order to defend the security and safety of the innocent.
Thursday, 20 June 2013 at 9h 52m 15s
Greg Mankiw and the Divine Right of Wealth
Greg Mankiw is a neo-conservative ideologue who pretends to teach Economics at Harvard University.
I say pretend because Mr. Mankiw is of the academic ilk that cannot separate personal philosophical
biases from the development of Economics as a science responsive to criticism and rigorous
investigation. Mr. Mankiw is the "goto" guy who writes academic papers that justify the elite view
of the status quo.
From 2003 to 2005, Mankiw was chairman of the Council of Economic Advisers under President George W.
Bush. In 2006, he became an economic adviser to Mitt Romney and continued during Romney's 2012
On November 2, 2011, some of the students in his Economics 10 class walked out of his lecture. About
60 to 70 out of 750 students participated. Before leaving, they handed Mankiw an open letter
critical of his course, saying in part:
"we found a course that espouses a specific—and limited—view of economics that we believe
perpetuates problematic and inefficient systems of economic inequality in our society today ...
Economics 10 makes it difficult for subsequent economics courses to teach effectively as it offers
only one heavily skewed perspective rather than a solid grounding on which other courses can expand.
... Harvard graduates play major roles in the financial institutions and in shaping public policy
around the world. If Harvard fails to equip its students with a broad and critical understanding of
economics, their actions are likely to harm the global financial system. The last five years of
economic turmoil have been proof enough of this."
The students concluded their letter by stating they would instead be attending the Occupy Boston
demonstration then under way.
Mr. Mankiw's smug and often snarky reaction to legitimate criticism is either arrogance or
marginalization. In other words: he's better than you, and the points you make (although true) are
insignificant to the big picture only he understands.
Which brings me to the point of this blog post. Recently, Mankiw wrote a paper that explains the
reason the top 1% (or top 0.1%) have a disproportionate share of
wealth (in the United States or the globe) as due to their individual societal contribution, or marginal
productivity. Someone is rich and wealthy because they contributed more to society relative to
someone else. Or in mathematical terms, the change in social contribution per something else (the
derivative with respect to ...) is larger than other members of society.
There are so many criticisms of this "great person" or "marginal production" theory by people more
qualified than myself. I
have included two good ones below by Matt Bruenig at Demos.org, and one of my favorites, Australian
economist Steve Keen at Unlearning Economics.
[SOURCE:Steve Keen | Unlearning
Economics | 15 June 2013]
[SOURCE:Matt Bruenig | Demos.org | 19
I find the individualistic notions of economics by faux-academics like Greg Mankiw both ridiculous
and deceptive. All individuals
want to take credit for all they have gotten in this life, while at the same time minimizing or
completely ignoring the relevant
environmental or lucky aspects of their ascent. People who achieve a high position also tend to be
fairly ruthless and psychopathic, because these traits have an advantage in a competitive
environment that rewards monetary gain at the expense of almost everything else. It's kinda hard to
feel "good" about earning millions at the expense of creating externalities like birth defects and
low wage dead-end jobs, unless you are either ignorant or completely ambivalent -- which is a nice
way of saying you don't give a fuck.
Since the default un-analyzed mindset for any individual is a self-centered point of view,
individuals who achieve high social status will also tend (or want) to believe they deserve their
position. This is where people like Greg Mankiw play their social role for the elite who need to
feel like they have the "divine right of wealth".
Mankiw is no different than the court suck-ups
who told the King that he has a legitimate line of royalty and has been appointed by a higher
authority to enjoy a disproportionate share of the wealth of the realm.
Wednesday, 19 June 2013 at 18h 3m 16s
What should be asked at the NSA hearings
This is what we all need to realize. Power can metastasize from even the most well intentioned
ideas. Given that the government has basically become the plaything of the corporate elite and the
plutocracy, the omnipotent surveillance state can be used for outside groups and interests. Someone
appointed to a position could get information, and then resign a year later, becoming rewarded by
the interest groups that benefited.
I don't know about you, but Mueller looks creepy to me.
What this means is more feds sitting in air-conditioned bunkers looking at various LCD screens of
video feeds from the drones they are using in a continuously increasing "limited"
Anyway, I got the blockquote below from a fellow who lives in San Francisco named Marc Perkel, and
it was posted on bartcop.
Congress is asking the wrong questions. I think if they really want to know what’s going on
they would give Snowden immunity and bring him back and get him to tell Congress what’s
really going on. But short of that, here’s what I would be asking if I were on the committee.
We now know you are getting information for the cell phone companies and major IT companies like
Microsoft, Apple, and Google. Are you also getting information from the banks? And if you are,
are you accumulating a database of credit card purchases?
Do you have the ability to include NSA code into operating system updates that would create a
back door for the NSA
to collect keystrokes, activate the microphone or cameras, read the file system, or modify the
operating systems of
Windows and mac computers or Windows, iPhones and Android phones?
If you have an NSA back door into our computers and cell phones, what do you have in place to
prevent China, Russia, Iran, North Korea, Anonymous, or Al Quada from discovering the back door?
If the NSA databases were hacked by China, Russia, Iran, North Korea, Anonymous, or Al Quada,
what would the security implications be?
Since Edward Snowden had the access he had, what is the security implications for an evil
Snowden who was selling this information on the black market rather that trying to protect America
(rightly or wrongly) from itself?
With an operation this large did you have any reasonable expectation it would remain secret? If
not Snowden wasn’t it just a matter of time before we found out for someone else?
Wouldn’t it be better to give Snowden immunity and bring him back here rather than risk that he
be captured by an enemy and forced to reveal what he knows to an enemy?
Because the NSA has extracted this data and is keeping it outside the source companies, doesn’t
that increase the security risks and exposure of sensitive information?
Isn’t our position that there are 2 kinds of people in the world, Americans and foreigners, and
insult to 96% of the world’s population? Does this not cause them
to respect us less and have no respect for our privacy when we have no respect for theirs?
Does our spying set a world wide precedent whereby other countries will have the green light to
spy on us because we spy on them? Doesn’t this put the whole world at risk?
Assuming it was inevitable that this spying would eventually become public, doesn’t it put
America in greater danger by alienating the rest of the world
because we are spying on them? Isn’t this just the same argument as the torture argument, that
we increase the number of enemies more
than we prevent attacks? Isn’t this just going to be another terrorist recruiting tool?
When you tell us that you are lying to us for our own good then why should we believe anything
Don’t laws that undermine and nullify the constitution, even if it’s for our own safety, make us
a nation that is no longer under the rule of law?
That secret courts and secret warrants making secret law make us no longer America?
The secret NSA court orders require companies to lie to the public about what information they
are giving to the NSA. If these companies
tell the truth they are punished. What the government requires its citizens to lie and punishes
the truth, how is that not an Orwellian society?
What America lies to the world and we have secret courts that require citizens and corporations
to lie, doesn’t that weaken America as a world power?
Doesn’t that send a message to the world that we are not to be trusted?
Doesn’t the appearance that we are taking the same kind of steps that one would take towards
becoming an Orwellian society create at least
an apparent threat to the world and undermine our international relations?
With this kind of information is there not an opportunity for criminals, companies, or our
enemies to blackmail our elected officials?
For example, Senator Vitter is tracked using his smart phone GPS to 5 hotels. One could search
for all smart phones in his vicinity to determine
what women were with him at the same time and cross reference them based on how often these
women frequent hotels in these areas to
determine if he is seeing prostitutes again. And that this won’t be revealed as long as he
supports some cause – such as NSA spying? How do we know this isn’t already happening?
Doesn’t the NSA already effectively have a gun owner’s database? For example, if someone joins
the NRA with their credit card online, buys ammo
at Walmart with a debit card, and has his smart phone with GPS in his pocket while going to a
gun club or firing range, don’t we know that are a gun owner?
And because of their GPS phone, don’t we also know where they are so we can pick them up at any
If we see government misconduct, like we find a CIA torture camp, and we want to report it to th
press, doesn’t the fact that the NSA
has the news media’s phones and email tapped discourage whistle blowers from reporting things
that need to get out into the press to protect democracy? Doesn’t this threaten free speech?
Since this kind of technology can be used to create an Orwellian society, what safeguards do you
have to make sure that doesn’t happen? Have you considered this possibility and fully explored the
Taking all of the above questions into consideration, how do you repair the trust we should have
in our government and how to we assure the “foreigners” that we are going to respect their rights as
citizens of the planet?
Ah, but someone made a bundle of money, got to become super rich, have a lotsa toys, and own scenic
acres of land with huge gated mansions and a swimming pool.
Only to get cancer and have to sell everything because the insurance company decided to screw you
with litigation and better lawyers than you can afford.
Saturday, 15 June 2013 at 16h 10m 10s
Why you should care about the NSA et al Surveillance
Because there are a lot of false positives. They make mistakes.
For example, when a computer programmer named David Mery entered a tube station wearing a jacket in
warm weather, an algorithm monitoring the CCTV brought him to the attention of a human operator as
someone suspicious. When Mery let a train go by without boarding, the operator decided it was
alarming behaviour. The police arrested him, searched him, asked him to explain every scrap of paper
in his flat. A doodle consisting of random scribbles was characterised as a map of the tube station.
Though he was never convicted of a crime, Mery is still on file as a potential terrorist eight years
later, and can't get a visa to travel abroad. Once a computer ascribes suspiciousness to someone,
everything else in that person's life becomes sinister and inexplicable.
Doctorow | guardian.co.uk |14 June 2013 ]
How many David Mery's are out there? How many innocent people do we improperly tag? How effective
is this procedure? What is the ratio of intercepted events to non-intercepted events? How
insulated is this from potential abuse?
I do a lesson with my statistics students involving a test that is inaccurate ONLY one percent of
the time for a rare disease. Because the disease is rare however, the one percent error rate of the
huge number of non-diseased people will produce a large amount of false positives.
Say a test has a 1 percent error rate for a disease that occurs in 0.1 percent of the population.
This means that out of 1,000 people, 1 person has the disease but one percent (the error rate) of
the 999 who don't will get a positive test result nonetheless, which is 9.99 or 10 people. One
percent of the diseased 1 person out of 1,000 will also get a negative test result because of the
so only 0.99 of the 1 gets a positive (1 minus .01 times 1) test result. Adding the 9.99 false
the 0.99 true positives and you get the total number of positive test results, 10.98 out of 1,000.
Notice that 9.99 out of 10.98 positive results is the percentage of false positives.
90 percent of the positive results are false.
Even if the error rate was 0.1% (99.9 percent accurate) the false positives would still be 50%. You
can't rely on data mining to find suspects. You have to use search warrants after the proper police
work, which is how almost every single terrorist was ever found or initially identified. Data
mining isn't useful for finding terrorists.
But the number one problem with the data mining aspect of the Surveillance programs is too many false
positives. How many lives will be ruined while we believe the illusion of security?
Saturday, 15 June 2013 at 15h 20m 21s
Glenn Greenwald 1 hour speech
You have to respect the knowledge that Glenn Greenwald has. He gives this essentially impromptu
speech and pulls out quotes off the top of his head. You can also tell that he is someone who is
truly strong in his convictions by the elaborate and honest reasoning he gives.
Of course, I'm sure he's not a saint, but if there is any dirt on this man, it will be out there for
them to find and broadcast loudly everywhere.
I suspect there isn't any, because Glenn has been good with civil liberties and the 4th amendment
for more than a decade. He is reliable where others (I'm looking at you Josh Marshall and Dianne
Feinstein) have shown themselves fickle and inconsistent on these matters.