The FDA and the Pharmaceutical Industry

The FDA and the Pharmaceutical Industry

we're delighted to have the harvard department government dan is the ally freedom Dasara Government at Harvard University and he's developed some of what I personally think is the most interesting work that we've had on bureaucracies and a disappointed political science and he this is related has done a book like that company should have had here to hold up that kind of shameless promotion is not possible anymore multibyte said all right excellent excellent book on the FDA Network and some seven projects doing forward thank you so thanks for having me I was invited to talk today about the FDA in the pharmaceutical industry is kind of a general theme so this talk will have basically two components in two purposes one is to kind of give you a general overview of the stuff that I've done in this area and to sort of pass along some general lessons including those in the aforementioned book that the Tony just mentioned and the second is to present some new results that we're working on with my research team at Harvard and that's the part so the first part is kind of in red here and the second part is after including the first part is largely published and so you should cite that this is the really the second part where I'm asking to be perhaps a little careful with the citation patterns of of what I'm conveying today so why is pharmaceutical regulation of interest to students of public policy to students a political science economics things like that well by comparison with a wide range of other industries there's actually much heavier governmental involvement in this industry both in the United States and worldwide so states and by which I mean they in states although sometimes in some cases as you've seen in California and Texas with some of the the bond issues and and referendum passed measures on funding basic things like stem-cell research states fund some of the basic research that goes into this industry most of the Applied Research and most of the money that is spent on pharmaceutical and biotech R&D is in fact private money but still it's fair to say that there's a kind of a complementarity and a kind of a mutual dependence between that work which is often built off of some of the basic things that are funded in part by pharmaceutical companies and private foundations but also in part by government agencies like the National Science Foundation or the National Institutes of Health the state regulates much of applied research what I'm gonna refer to as the conceptual power of the regulatory state basically the ability of the government to define the vocabularies methods and concepts that people use in research and so much of the history of the pharmaceutical industry in the 20th century is actually the history not so much of science developing exogenously from government regulation but science often developing endogenous lis within government regulation I'm going to show you one example of that okay and so just for instance if you want in the United States to test a drug and to basically transport that drug across state boundaries you have to get an exemption from the FDA which is called an investigational new drug exemption or IND alright and that is basically a prerequisite in order to engage in medical research with clinical subjects which is to say human subjects in the United States by virtue of the diffusion of those rules worldwide that is now basically a global order of regulation right the state is also a veto player over R&D so if you you know if you're a for a car company and you design a new automobile for the most part the government is not sitting there at the end of the law mine with the ability to veto whether that project enters the marketplace or not all right I mean you can talk about different ways in which a wide variety of markets the state enters licensing land use permitting environmental permitting things like that but again here it's quite strong in its product specific it's not licensing firms so Pfizer in an in in some way or Merck are not licensed by the FDA although they're kind of certified with the way they produce drugs but each and every drug that they would wish to introduce to the market and on which they would seek to generate profits has to be approved by the FDA all right and finally less so in the United States but increasingly worldwide once those products are on the market the state also regulates their post market life that prices they can charge all right which we see sort of in Europe say for instance in the United Kingdom through the National Health Service or for that matter in Canada but also I'm also the way they're distributed so there's been a couple of recent developments at the FDA with the distribution of opioid related drugs and the FDA basically trying to induce pharmaceutical companies to develop less tamper resistant drugs so that hydrocodone and oxycodone based medications can't be kind of mixed into a soup and that is more addictive right now the general theory that I've been working on which kind of functions as a background to this is what I call the theory of approval regulation and the idea is is that the state is again in this capacity of being a veto player over R&D and that there is kind of a simultaneity between firms which seek to kind of develop profitable investments but those investments are kind of the value of those investments is known only with a great degree of uncertainty and that the firm is also regulating those investments but again knows perhaps even less than the firm does about them and so it's a world where basically companies are trying to bring these products to market they are not sure that their product is profitable all right they need to impart test the product through the Rd process to be sure of its quality or at least more information about it but again there's this Vito player right now so the regula regular is a decision maker under uncertainty which I described as kind of stochastic all right part of what we do is that we then model and estimate statistically a set of political constraints on the regulator and when I refer to the regulator just think FDA um in its approval behavior right so some of the work that we've done and this is work that other people have done to is to describe how even in the absence of like political capture or a political protection large firms are often gonna do better under this process why in part because they're more familiar to the FDA all right people who enter certain market niches earlier often do better not begin because there's some capture dynamic but because the regulator can approve drugs for say a new cancer therapy as a way of kind of throwing a bone to patient advocate groups and things like that so this is summarized in some work that I've done with the title protection without capture where one gets protection for larger producers older firms and older or first entrance to a marketplace without there being any degree of kind of political purchasing or bribing of the regulatory process and then finally in some other modeling that I've done with Mike King who I referenced on the first slide we've looked at the endogenous of R&D decisions and regulatory approval and there's been some subsequent development where we've looked at what happens in this world to consumer confidence so basically people coming into a marketplace in which there's a certain degree of screening so in theory bad projects products might be screened out the products that you do have that enter the marketplace there's a lot of data produced about them such as randomized clinical trials summaries of those data make it into the label and so the question is what happens to consumption and then there's a more general model that seems to be applied to more to antitrust by to economists otaviano and Wilco Grenon all right but one problem do I have this hero I don't have that one problem here is that we actually we have a lot of data in theory about this problem which is basically how the regulator's decide we have a lot of data and theory about how firms develop their products in R&D we have very little data although now again just some emerging theory on basically how R&D and regulatory approval strategies respond to one another so I like my impression was that the FDA approval was their more objective process and you know clinical trial shows there is not much subjectiveness or description but these two researchers that are you suggesting that you know files for some diseases or something even if they don't show they have a different special yeah so if if it were really that objective I think you'd find less disagreement within the FDA and less disagreement within the advisory committees that offer their counsel to the FDA than you see so I guess I would kind of disagree it's not that I basically yes the process is very scientific yes there's a lot of data that informs it but science number one doesn't eliminate the uncertainty and sometimes the science generates more controversy than in fact reduces so I think the process is is shot through with science and in fact rigor I mean what we know about these products coming into market probably is greater than just about that for any other sort of form of industrial organization that said sometimes that information can generate controversy and subjectivity for instance we'll know a lot about these you know these products because they've been tested and randomized clinical trials with thousands of patients right but from those trials we might get a safety signal that suggests that well wait a minute sometimes after 18 months there's some hepatic toxicity right that's developing in the liver right how do you interpret that do you interpret that as something which is so important that we should thereby reject the drug or something that we should they are thereby or thereafter attach a warning to the label right that's a controversy which actually is generated by the scientific process and which is actually not so much reduced by it is that kind of okay so the book that I've done and so if Tony had had the the hard copy here he would've been able to give you this was published in 2010 which tries to unify both historically theoretically in a conceptual manner and empirically a large number of these observations all right um and so let me just give you two before I get to the sort of newer work let me just give you two kind of lessons from this book that I couldn't sort of talk about the first is is that it's commonly thought that basically the way that the FDA evolved was in sort of three kind of crucial enactments number one the 1906 Pure Food and Drugs Act which gave to the FDA actually was a Venna Bureau in the US Department of Agriculture power in interstate commerce to govern food and drugs in 1938 it got this pre-market approval power but only for the question of safety not whether drugs actually worked all right and then along comes the thalidomide tragedy in 1962 which essentially didn't occur in the United States because this woman Frances cowsy held up this drug which was contra gand the lid amide which made its way into Germany there were thousands of birth defects things like that but the usual story is is that only in 1962 after that tragedy in Europe did the FDA begin to regulate efficacy and in fact people have used that sort of before-and-after comparison in a wide variety of studies and economics and political science to try to essentially estimate what the effect of efficacy regulation is versus safety regulation well basically one historical lesson of this book with a lot of time spent in the FDA archives as well as pharmaceutical company archives is in fact that the FDA was regulating efficacy more continuously in kind of an upslope from the late 1940s all the way up until 1962 so there's no sort of tight boundary pre and post right so here's just an example Erwin Nelson who's the head of the drugs division in the FDA in 1949 gives a speech to pharmaceutical company representatives in which he says look we want proof of safety that's what the law says but we also want proof of efficacy this is one of those cases where simply by communicating things in a speech you know a federal agency manager or bureaucrat or regulator can often go beyond what the law says not in a way that's illegal right but in a way that's kind of non statutory right and so nowhere you know did the FDA's rules say that you have to prove efficacy to get it your drug approved but increasingly in speeches they're trying to you know giving this message and if you look at sort of the trade journals during this period industry trade journals where they're talking about you know what's the best new investment in the world of chemicals pharmaceuticals drugs foods they're basically complaining that the FDA is making a lot of these kinds of statements like all right now we don't know what the criteria are because we thought it was safety ten years ago in 1938 but increasingly it seems to be efficacy and if you want lots and lots and lots of those quotes with lots and lots and lots of sites consult Chapter three of my book which is about 100 pages long all right too long but it's got all that data available for you as evidence also of this basically the FDA began to use refused to file or RTF judgments which is to say we're not even going to review your drug application unless it meets certain minimal criteria and I've sort of listed those here and this was a draft Federal Register document in 54 the new drug application form was finalized in 1956 that's five to six years before thalidomide hits and again the drug advocacy amendments were passed and it says an application just gonna read this for you may be incomplete or may be refused unless it includes full reports of adequate tests by all methods reasonably applicable to show whether or not the drug is safe for use that was a way that they enabled efficacy regulation by saying not just safety in terms of toxicity like do you explode when you take the pill but safe as used right and that was a way of getting into how was the drug going to be used in what purposes and with what effects the reports ordinarily should include detailed data derived from appropriate animal or Butler biological experiments and and reports of all clinical testing by experts those experts must be qualified by scientific training and experience that was code for you better have a PhD in clinical pharmacology on your team otherwise we're not even gonna look at your application alright and it should include detailed information pertaining to into each individual treated including all these variables results of clinical and laboratory examinations made so if you took a blood sample from this person at the beginning and or in the middle of a clinical trial the results of that had to be available on paper not just in sort of a summary statistic and a full statement of any adverse effects in therapeutic results observed right so you have to tell us the therapeutic results in order for us to even look at your drug application all right this again was six years before the little might occurred one of the things that the FDA was doing about three decades before it occurred in Europe was literally getting the raw data from all these new drug applications what happened in Europe was companies would send statistical summaries from their clinical trials often highly observational not randomized in the US and this predates the lid amide you would not only have the raw data in the sense of the numerical data set you would have all the paper data from which the numbers were coded and they would literally go back and recode and examine the sensitivity of assumptions they were literally you know decades ahead at least in terms of statistical methodology replicability of where europe was at that time if you actually look at the approval time distribution how long did it take from the time that a drug was sent in for those drugs which is approved we're only looking for drugs which were approved here how long did it take them to get approved okay you see that in the early 1950s and these are quantiles in the approval time distribution so this is the time by which down here 25 percent the first 25 percent of the drugs are approved the first 50 percent of the drugs are approved the first 75 percent of the drugs are approved and here 90 percent of the drugs are approved right so this tells you something about if you will detail or the outer tail of that distribution right if you look in the early 1950s it's very quick and in fact the statutory standard is they're supposed to be approved within six months all right or reviewed within six months so if approved then approved within six months but you can see here a sharp uptick not only in the median but also the tails right whereby by 1960 before anybody knows what the lid amide is right before there's any idea about officially adding efficacy the FDA is already at the median all right going through its the Congressional standards which we're not binding right but at least were recommendations now is this proof of efficacy regulation no right but is it consistent with the story that the FDA was getting more stringent during this time period it's worth keeping in mind that if you just estimate this in a sort of aggression model turning out like basically you know how long does it take the FDA to approve these drugs and you control for the amount of staff that the FDA had at this time right the effects get stronger not weaker why because actually the FDA staff was tripling during this period so if you control for all those things this is it's pretty clear that this is something other than backlog and or resources right and again I'm just showing you summary statistics here this is not just from an estimation here right but and again this is not efficacy per se it could be a whole bunch of things but basically it's consistent with the story the procedural story that you can tell elsewhere all right second lesson so that is if you will kind of gatekeeping power right when we talk about the veto power the gatekeeping power that the fda has over the marketplace this is one form of it and they get to define what standards are used separating the wheat from the chaff right and in so doing they're actually able to define other kinds of standards so if you read the financial pages say of the Wall Street Journal or the New York Times and you you know refer to a biotech stock right something you're kind of interested in you'll often hear this you know okay you know for Aalto Pharmaceuticals had a key it promising for non-small-cell lung cancer that failed in Phase two trials you might ask you said where does this phase 1 phase 2 phase 3 stuff come from right well again this is a general lesson of the book consult chapters 4 and 5 if you want more but basically this is a creation of the regulatory state imposed upon medical research and scientific research not the other way around alright and there's a long history that goes into literally when these phases began to get drawn up alright if you look at for instance and the key rules were written in 1963 alright there are a few phase trials before that we're actually sanctioned by the National Cancer Institute in fact most of them run by the National Cancer Institute so the story of the development of phased experimentation the idea that one not only runs a test for a drug but you run one set of tests successful passage through which becomes a sufficient hurdle to go to the next set of tests sufficient passage through which becomes the sufficient hurdle for the third set of tests right this idea of sequential experiments right that is a regulatory imposition not only on the pharmaceutical industry but in fact on the entire medical industrial university complex in the United States and in fact worldwide every human clinical trial now that involves a drug all right of any sort is essentially going to be classified into one two or three now there's four and there's technically a zero and you know but but if those are just further glosses on this basic structure original documents and sites they're again some ideas about this were thrown around by the National Cancer Institute as well in the late 1950s but the original idea for this idea of sequential experimentation actually comes out of pharmacologists animal pharmacologists in the 1940s looking at how to test for the safety and nutritional value of different feeds for livestock and part of what they're interested in is what's the acute effect and what's the chronic effect and if you think about phase one and Phase two it's kind of a development from that you're looking in phase one it kind of alright people explode when they take this pill do they you know basically fall over phase two and phase three are what are those longer-term effects you're moving from acute to chronic right well again this is not only or not purely endogenous to science in fact if anything it's imposed upon science and if you follow the pharmaceutical industry you'll know for instance that if a company is not publicly traded and it's getting its money from venture capital the people in that company are often paid by benchmarks right have you met a certain benchmark then the money comes in well the benchmark in a lot of these cases which is by the way the money that people make in in the biotech industry is often the successful completion of a phase so literally the way that pharmaceutical payment contracts are structured in the biotech sphere for those companies that are not publicly traded is in fact shaped by these regulatory categories so it's not simply conceptual power in science it's conceptual power and science that shapes the structure of industry and payment contracts so to if you want to look at where the big movements occur in asset prices for pharmaceutical companies it's often on the announcement of phase 1 phase 2 or phase 3 results often are also approval advisory committees and things like that so the major pivots for stock prices for those companies that are pharmacists are publicly traded also observe at some level this conceptual structure it's been a very powerful it's a simple idea right let's just set up a set of experiments in sequence in seriatim but it's affected not only science everything that goes on not everything but most of the things that go on at the Health Sciences Institute here but it also affects the structure of business right again read the book if you like more on that so now I want to shift gears a little bit to talk about a claim that's commonly made about pharmaceutical regulation and innovation and sort of here's the more speculative part of the talk and also the part that might be more relevant for pharmaceutical public policy excuse me communities Serban numerous claims made about the effects of this kind of regulation on innovation what do we mean by innovation the number of new drugs particularly new molecular entities molecules never before marketed never before used in widespread treatment in any other capacity and the claim has often been made that this regulation has reduced those that innovation not necessarily by the way in a way that's net cost beneficial negative because you could say well look we're getting rid of all these safety problems we're getting rid of the crap could be that it were better off but the argument has been nonetheless an observational argument an empirical argument that in fact after the imposition of this regulation things went down I'll get to that in a minute so claims have been made comparing things before and after major laws including some of the work that I've done claims have been made internationally so there was this old literature called about the drug lag in the 1970s about how these many of these drugs were reaching England in particular in some other countries in Europe before they were reaching the United States prickley with things like beta blockers cardiovascular treatments right the claims again are usually about reduced innervation although there are arguments that go the other way and say actually innovation or the larger sort of properties of the health system are improved that sort of go off the lemons argument in a curl off the argument sort of loosely stated is well once you start getting rid of quack cancer treatments or once you yank tranquilizers off the market as the FDA did in the 1970s you start to improve the market for cancer treatments because the bad stuff doesn't crowd out the good stuff all right but again these are just a set of claims the problem with a lot of these claims is twofold and I'm gonna separate what we usually refer to as in dodging a into two census year strict and dodging a 'ti in the sense that basically regulation often responds to patterns of economic activity which themselves respond to regulation right that's the endogenous can model and that I have modeled with Mike ting right so in approval regulation all these things coming to market right only are you know the FDA can't regulate can't or at least can't sort of make a decision on something that that hasn't been submitted to it right but firms and develop develop and submit according to their expectations of regulatory behavior and those expectations are probably correlated for what it's worth with a lot of other things that change around the time of regulation so if you're looking at the late 1930s early 1960s a wide range of scientific changes going on in terms of pharmacology applied chemistry and things like that alright the other problem is non-random assignment which is the usual thing we care about in these kinds of questions right I'm separating that from in dodging a ax T began to get because again in dodging a T is something at least partially we can model non-random assignment I don't know everything that might be correlated with the application of regulation in the New Deal in the early 1960s early in the early 1990s but suffice it to say if our research design is premise Tapani before and after comparison well lots of things might be you know correlated with that right so here's an example from one of the most famous studies of Sam peltzman on the 1962 amendments and so what he did is he looked at 1962 which was Wendy's efficacy amendments passed and he said well look the actual number of NCES which is actually the which is this series right here went down now if you buy his production function the way he sets it it shouldn't have gone down that much it should have stayed higher and so he has a counterfactual which is the higher one here and the split between these two functions occurs in 1962 and he wants to argue that difference after 1962 can be attributed to regulation and he finds or claims and other work that the cost of this is not made up by better therapeutics right now this is a pretty influential article and to give him his due this was published in the 1970s but one might worry about essentially basing policy on a 14 point time series followed by a ten point time series right and and estimating two different production functions there but the second problem is is that this isn't really kind of a treatment or an intervention in any way that we can plausibly call experimental right and again this is where I think an historical perspective actually helps to matter for one as you notice the sort of new chemical entities are falling from a peak in the late 50s early 1960s before 1962 happens and perhaps my chapter and some of my work on the application of efficacy regulation in the 1950s might explain that but at the very least we don't have a clean before control after treatment kind of world here right if in fact the numbers I was showing you earlier that basically the FDA is beginning to regulate efficacy here and we really can't trust a lot of the kind of judgments that were making about complaining or comparing things before and after a given date again to be fair he was writing something three decades yeah so so so I think mine could explain that right in part there's two other problems here one is he doesn't nor do I control for industry concentration there's some emerging evidence from the literature that actually suggests that one reason we've seen a little bit less innovation in recent years is precisely because of merger and acquisitions activity I can reference that separately and that was occurring heavily in this period as well now you could say well that's endogenous to regulation because people are facing a tougher regulator they want to develop regulatory affairs departments get big to basically be able to handle all this that's quite possible it's tough to kind of disentangle and sort that out I agree actually that if we're looking for the reason why we come from this rough mean down to this rough mean probably that smoother regulatory function is probably a plausible candidate right but the point still remains that then a before-and-after comparison using 1962 is not valid right okay so what to do well here's where we have an idea and this is a story that's actually taken from in part the first chapter or the introduction of my book reputation and power but I'm repeating it here and actually talking about some features that I don't talk about in the book so you may know of Genentech it's kind of darling of the California biotech industry it's now a quite big and profitable firm goes up and down but it used to be a tiny little firm and it had a very small drug called tissue plasminogen activator or activase and it submitted it to the FDA and was quite confident in fact that it was going to be approved right but a food and drug administration panel in June 1998 1987 excuse me basically said no voted against approval of the drug right and basically it wasn't and it's important to keep in mind when the FDA says no to a drug it never says we will never accept this molecule ever right they wouldn't even do that for cyanide I mean legally they can't what they say is and it's kind of like if you're an academic and you submit you know papers to journals it's like getting an endless R&R again and again and again and again without the certainty of ever getting an approval right so sometimes when the journal editor comes back to you and says look next time I don't give you an up or down decision the FDA never says that and that's actually a huge source of complaint among pharmaceutical companies like give us an if/then statement so that if we provide you this evidence we're gonna do that now I with some work I'm doing with a game theorist and another work I'm doing with a Nestorian we're actually trying to tease out why the FDA follows this kind of strategy of ambiguity and the difficulty is is it's very reluctant to kind of commit to a certain model of saying alright if you do this then we'll do this because then they feel that the firm's or other firms can number one game that and just basically come up with a weak satisfaction of the if part of the hypothesis and second that they're setting and this is I think the real reason they're setting implicit and sometimes explicit precedents for other firms and that's the other reason they do it I'm not saying by the way that's good policy I'm just saying that's the rationale I think that that we think I was going on but this was bad news for Genentech alright um this happened on a Friday and if you follow government agencies particular in Washington they often announce these things you know after the market closes this was one such example but when the market reopened for trading on Monday all right Genentech a stock dropped by about a quarter and about a billion dollars vanished just like that and so this is kind of interesting for two reasons one there were kind of surprises to this right a lot of people did not see this coming including a lot of people who had bet a lot of money on Genentech not just people at the company itself but Genentech was publicly traded right so and you can insert if you want your snarky reference to the Romney victory party in Boston here but they actually had planned a company executive victory bash right which wilted and I just wouldn't be able to write this as well myself into a combination wake and strategy session try that sometime after your next a professional difficulty okay and then the other thing is there's kind of if you will appear or alter affect a lot of other firms are looking at this and saying oh crap Genentech just got shot down now what are we gonna do and so here's one of these people quoted anonymously it's like well wait a minute now the FDA is kind of changed the ballgame here's something that we thought was a sure thing you know they've kind of raised the bar or we're not sure where the bar is so you see what we're getting into so here's the idea all right it doesn't solve every problem that I just talked about but it gets it how to assess the effects of regulation or regulatory decisions on innovation we're gonna use events like this they come with a certain degree of surprise we can measure that surprise in a general equilibrium financial market right we're then gonna use those surprises as weights so every time the FDA makes one of these decisions it's gonna be weighted only to the degree that it moves the market we're gonna filter that price to try to get rid of other contaminants right and then we're gonna use that essentially to affect what other firms do not what Genentech does after it gets its drug shot down but what other firms do with that okay that's the strategy and by the way I think this is at some level consistent with the larger story that the book tries to tell because gatekeeping power and for those of you who are in political science who study Vito's right the power of the veto is not simply the power to say no to something that comes your way it's to induce everybody else who would send something your way to begin thinking twice about whether they want to send it in the first place right so geek gatekeeping power is not simply the power of decision it's the power of induced anticipation so there's a set of complicated effects here and so for you know for purposes of statistics what I'm presenting to you is an average of across all of those so what a statistician would call an average treatment effect of this that is going to combine both the response to the FDA right it could be the higher bar it could be FDA uncertainty and it's going to combine the fact that other people might see opportunities which means that if anything I'm probably under estimating these effects upon innovation right because what I'm going to show you is an average that's a composite of all those things but one of those composites is probably I can't say for sure because we'd have to net this out and we're in the process of doing that but one of the the the building blocks of that composite is probably positive which is to say other firms might see an opportunity here and might actually continue with their development projects not pull them back I do tend to think actually that the way that most firms respond to these things is that the regulatory effect washes out any like market opening you see that quite commonly because the bottom line is all these other companies right who would wish to get into the market who say ah Genentech might no longer be there but if they're gonna be where Genentech was take up that niche they're gonna have to pass through the regulator – right so again so what you're saying is very interesting and useful and basically it's going to depend on defining the set of competitors quite exactly what's the therapeutic marketplace or niche what's the mechanism of action and we're doing that you know in a further extension to this but right now what I'm giving you is essentially an average – I'm people drop in people drop midstream all the time they simply not on the result of that external factor external factor by the way doesn't have to be regulatory it could be we had a bad budget shock we had a you know a new sort of chief financial officer come in looked at our portfolio of active projects and said we don't like this and why if you're gonna make that decision to kill why wait until you know something is is done if you think you have enough evidence already and you're just gonna you're gonna make a business decision to say all right stop this clinical trial now there are issues about human subjects protection and things like that that might extend the clinical trial a little bit further in today's environment but again this does happen mid-stroke doing well in phase one or phase two and still being pulled oh yeah absolutely now that's anecdotal and that's kind of hard to sort of quantify for doing well in phase one and phase two we've got some ideas about how to do that but plenty of examples so when you look at these events are you looking at events where the FDA decision was a surprise within this genetic is it seems like they they actually showed that their drug reduced this particular enzyme or whatever thing it was didn't and this is important that means improve survival and every even by the document verses a clinical trial where it just failed because gravity knows we're not looking at those because those would have happened anyway right so we're looking at cases where it's the regulator associated with an event and we're using the stock market shift as an indicator of the surprise all right and the idea here is if we're trying to sort of be kosher with our statistical estimation we want something that's both non anticipating randomness and to conditionally not correlated with all the other things that we're worried about that might be correlated with that right so there's not I don't have a background model on here today but basically here's the kind of approach that we're talking about so imagine that affirm is choosing dynamically every moment okay in time DT if you will between a certain drug that it's developing and this is by the way not Genentech this is you know Genentech's competitor right between a drug and a safer investment which gives you a known return which we're just going to call a put option all right and it values this the value of its investment is stochastic and it basically is a function of an initial state followed by an exponential X all right so the this is basically always positive think about this is kind of analogous to a stock price right and this X is gonna be a what I call a levy process what we call a levy process alright and that means it can have these more continuous things like a Brownian motion or Wiener process it can also have jumps which are these kind of very discontinuous up-and-down movements all right now if I give you the following and there's a feint I'm just gonna wave my hands at French mathematician Paul Levy if I assume the following things independence of the increments from one another so given any given history the next movement is independent of what came in the past right stationarity alright so basically the idea that the expectation of these movements at any time is itself moving in a stationary way and the continuity when I mean continuity in probability of the increments obviously there's discontinuity in the jumps itself but the probability function describing them as continuous there's a something called the levy decomposition theorem and a set of other results that basically anytime you make just these three results you always get a levy process the levy process in turn is essentially described by and I'm just gonna wave my hands I'm being kosher to give the kind of full equation here but it's a linear trend alright which could be zero right Brownian motion which is this kind of you know little thing butterfly popping around and then again I'm just doing this to be kosher because there's a knot at one that is again in the kosher theory you can't integrate over it jumps so all this stuff here is just discontinuous jumps all right so every levy process is a sum of Brownian motion a trend in jumps and each component the trend the jumps and the Brownian motion are independent of one another alright so the idea here is this again what we want to do is focus on these jumps again just I'm gonna wave my hands at all this kind of you know lovely math and say that's jumps what's left over is something that at least in a reasonably functioning general equilibrium financial market is already priced in all right and then noise right which means actually there's every time we observe one of these jumps a little bit of it is due to this right where so we actually have a little bit of measurement error but we can plausibly claim that measurement error is itself random or not so that's what's happening for a given firm but maybe the firm and this is again one of Genentech's competitors okay so let's call it um you know genome therapeutics or something right maybe it's decisions depend on its observations of another firm like Genentech right so that the value alpha is a function both of its own product but also some function of another product not its own whose success or failure and that includes success or failure in the regulatory domain tells that firm something useful about its own product right now we don't see that other product as analysts right as somebody crunching the numbers I don't see what's going on with that other product but I do see a stock price that's based in part upon that product and what I'm just gonna focus here is on the negative jumps and I'm gonna do the same levy decomposition I did earlier right if again it's that has these properties I can reduce it to linear trend noise and jumps I'm sorry yeah noise and jumps alright so those jumps in theory and I we can actually test some of these things should be not anticipable you can't tell they're coming ahead of time one sufficient but not necessary way of getting there is just to assume a perfect market if you could know you'd make a lot of money therefore you would make a lot of money and all that information is already priced in right but again it's also if not anticipated given information up to that point in expectation with other basis of firm information alright so I'm gonna make the claim this is probably ran it's not an experiment there's you know plausibly random so here's the idea the research design is we're gonna use Wall Street Journal stories on FDA rejection request for more data for drugs under NDA submission but not yet approved all right so we're gonna take these stories we're gonna compute either the day those stories come out the day the FDA makes the announcement or sometimes the cup or the day after if that's the trading date it's relevant like the Genentech case just the one day shift in the asset price for that sponsor the stock price right you could say we should do more and we've done a little bit of that and we're you know looking at other filters but the idea is we want to capture only what that event had and not you know some other event that might happen like somebody got fired you know somebody came in there was a you know some new sales figure that came in we want to capture only that event we apply that as a predictor to whether all other firm's development projects which is to say all the thousands of drugs they're developing happen to get dumped in the months following or continued okay so we observe from the early 70s to December 2003 and this is actually for the most part 1987 to 2003 and most of our or 1985 most of our analysis is focused in those 18 years about 187 of these right and if we analyze basically what's the correlation of those shocks right the the shock in the stock movement with a set of things that we can measure we tend to find not much correlation so do the shocks get bigger over time no they don't get bigger or smaller are they correlated with the beginning price because one of the ways we're measuring these things as the percentage change so you might be concerned about a denominator effect again 0.05 correlation not statistically significant are they partially correlated with the size of firms that are developing drugs at the same time again they're not are they correlated with the general movement in the stock market that day well not surprisingly yes because on the same day it could have happened the Labor Department could have come out with a report that said unemployment is going up or down it could have been some major market shift it is correlated although not a ton and one might but one thing we can do in which we do do and I'll I can describe this as we essentially purged our estimates of this general movement so what we're looking at is essentially the specific firms movement purged of the general market movement right and we're working on tests of whether these satisfy levy properties so some threats to inference might occur let me just sort of give you a little bit of the soft underbelly of the research design here okay what finance specialists will call volatility clustering is a possibility and that's the idea that well you can't predict whether it you know the stock is going up or down on a given day but if the stock is moving around a lot one week it's been shown that it's more likely to move around a lot the next week so there's first moment independence if there's not second moment independence and stationarity in many cases all right and we are again still working on a purge again what that would do is not so much change if this were a problem would not so much change the sort of the the validity it would change the the interpretation of our estimates from one of sort of the FDA is changing its bar raising its bar or lowering its bar to the FDA is becoming more uncertain but that's a significant enough change in interpretation that we want to you know track that down the second course is the FDA does not report on all of its negative decisions so you actually have to go to news services all right including the Wall Street Journal or others to track when the FDA hands out a negative decision the reason is it's a complicated exception to the Freedom of Information Act the FT if you ask the FDA is a drug from Pfizer currently under review at your agency the FDA cannot answer yes or no and that is considered proprietary trade information you cannot request information about that application under the Freedom of Information Act again because it's proprietary trade information whatever whether that's a good policy or bad it sticks right so we actually have to look in the news for reports of this sort and it could be that only you know surprises of a certain magnitude or are likely to get reported that does not change the fact that the day before their reported they're not anticipable right but it might change something about the distribution of what we're observing and then finally there is someone who actually knows that these decisions are coming right and that's the regulator or the regulator's themselves right so you might know of Martha Stewart in the time she spent as a guest of the state I hope she doesn't watch the YouTube here she was actually brought up on charges of insider trading but actually got convicted on charges of perjury in that investigation Sam woxall was also I believe indicted I don't know whether he went to I don't know the exact stir but he was also part of that case here's a case where an insider a chemist at the FDA all right knew that drugs were gonna be turned down or delayed right often focused on small biotechs right and bet on shares falling after negative decisions in sold shares to avoid losses so exactly the kind of thing that were occurring if this occurred a lot all right like this was an everyday occurrence and people like this didn't get caught that would be a big problem for the research design I'm presenting you because essentially it would mean that a certain part of that surprise is essentially priced out or priced into the market before it occurs because of all this kind of trading right reason I don't think that that's but I'm presenting it because it is a concern the reason I don't think it violates you know the sort of validity of this research design is twofold first off these people do get caught mister and yang is now serving five years in a federal prison right second the extent to which they can make money off of this right is limited by the degree that if they traded so much as to cause me as an analyst problems they would be all the more likely to get caught so they can make a lot of money for an individual right they can't make so much money that they begin to really change the stock price if they do they're far more likely to get caught right if there's a couple days before this and you see like two three four percent swing in a stock price due to one individuals trading even the SEC I'm sorry but it's the SEC has been getting a lot of criticism lately you know a seven-year-old with a spreadsheet would probably be able to pick up that kind of activity and detect the insider trading okay so here's what these asset price shifts look like this is a fraction change so if you're looking for percentages just multiply by a hundred all right so the mean is about you know at ten twenty percent drop in stock price after one of these things occurred sometimes you know there's just not much of an event and there's so these are the kinds that get get essentially weighted as zero it's as if they don't occur those rejections don't occur some of them are you know companies losing 75 percent of its value now one of the things you might be concerned about again is that some companies might more be more likely conditioned on this happening to lose more of their value than others so one of the things we do in addition to using the raw value purged of the general movement is also to binarize the treatment which is to say let's have a cut-off say right here all right did the stock price fall more than this amount as opposed to that amount for what it's worth actually that does reduce the error and the models that we estimate quite a bit all right so that might suggest that there's a lot of you know extrema bouncing around this distribution right but we do we do both alright and the other thing we do is essentially we observe a list of thousands of drug projects that are undergoing development at a given point in time and essentially we if you've used Cox models before we essentially use a Cox model of duration how long does it last before it's abandoned right but it's a little different in that the analysis is conducted not only across drugs but within drugs and the idea if you're sort of into kind of epidemiology is this is kind of a within subject treatment all right so we're controlling for all the features of the drugs themselves that are under development the non Genentech's if you will right but we're looking sort of what happens within those drugs as a supplement one of the things I'm gonna do is use a linear probability model all right which is basically zero when the drug is continuing one when it gets abandoned all right just gonna run a simple generalized least squares regression on that and include a fixed effect for each and every drug which is namely fifteen thousand of them so it's gonna highly saturated model and again that's gonna turn this into a differences in differences estimation and that's also going to be a within subject treatment all right so here's what it looks like I'm sorry here's the data so if you will the dependent variable is we want to find out whether companies are moving on with their projects toward further testing or submission to the FDA or whether they're ditching them saying enough of this right we have about 14,000 projects under development between the mid late 70's and december 2003 and these are followed monthly so we've got about a half a million observations in our database the coverage is better after 1987 because this is a proprietary database produced by pharma projects and I mean this is a private company that's been following the market for a long time that aggregates a lot of these market reports the coverage is Ginn gets better and so one of the things we want to do is say alright let's only look at the data after a certain amount of time and then change that just to see whether our results still hold up one limitation and I'm started trying to get a grant for this is this is all before the Vioxx tragedy which you know by some estimates you know contributed to twenty thirty forty thousand excess deaths things like that there's an argument that the FDA got more procedurally conservative after the Vioxx tragedy that I think needs to be tested but we're not gonna see that in these data we have two different measures of abandonment right one is when the company just says we're done with this and they come out with an announcement right often companies don't want to say those things in part because they want to sort of keep their options open and things like that so we have an implicit one which is where this database reports no development reported right once that happens for two years we go back and code it from the time it originally started being coded as such and say the drug was abandoned we use each of these alternatively and then we combine them alright so that we're not dependent on given one measure we allow the effects then of these shocks to be generic which is to say applying to every firm or applying to a firm which is a rough competitor or an entrant into the therapeutic niche say cancer drugs central nervous system drugs cardiovascular drugs in which the bad events or the negative news for one company happened all right we're defining this class very broadly this gets to your question about the competitive effects so one of the ways we're gonna do that here and we could do it much more narrowly with kind of refined data on the mechanism of action right now I'm just going to use the division structure of cedar now in part cedar by the way is the Center for Drug Evaluation and Research it is the fda bureau that makes these decisions on the drugs and so one reason you might we might want to do that is because if the extent that these folks are making inferences about the FDA and saying oh my goodness the FDA is getting much tighter they're not just making you know a judgment about the FDA generally but about the particular rule you know the particular decision makers in the oncology division or in the cardiovascular drugs division who may have changed their standards and said oh no no you know P less than 0.1 zero is no longer statistically significant we're gonna say that's P less than 0.05 or you know we're gonna you know demand you know another different kind of clinical trial with another different kind of treatment arm or control arm before we you know send something on to the next stage we proved it right they might be making in other words decisions or inferences not about the bureau or the regulator writ large but about sub regulators within that bureau right which is one way of actually thinking about possibly a way of kind of quantifying agency reputations and sort of D compartmentalizing or compartmental and decomposing the agency writ large as changes at the FDA this would just be signed a big surprise so we're doing the clinical trial for a certain drug and you know you were hoping it'll work but it did work and you know that change science and stock prices plummeted for the stuff in because everyone thought it would work but it didn't work and it's got nothing to do with how FDA valuated but in some sense it's a mixture I don't know well so it's always true I mean so here's the problem right is is that is that every regulatory decision is a decision about the merits of a given drug right now the if it's if it's a decision about the merits of a given drug right then we should clearly see a within firms to say Genentech got this bad news about its drug they should you know drop it there it's not clear that that logic extends to everybody else including outside of the therapeutic area right so that's that's exactly why we're doing this if you're right we should observe a lot of class specific effects also be like financial shock the New York VC and genetic stuff plunges you're like I'm out of all biotech I'm investing in colleges yeah you're out of all biotech precisely because the FDA ruled against you no but it's not because the FDA rules because the science was bad that you know there's a lot of hope that biotech is gonna produce great medicine in genetic eye on things I changed my expectations about biotech new camera so this is bad science I need I first of all I don't think sort of a general equilibrium market that's gonna happen I mean basically did especially with a publicly traded company right there's enough other people to say look there's you know a possibility here and you know it's possible there's gonna be an overreaction and things like that to the extent that it's about you know purely you know it's picking up purely like a scientific development first off that's not inconsistent with my story all right basically this is you know the science is being produced but the science is being produced and judged by the regulator by the regulator's Advisory Committee but so you can view this as a scientific revelation in many cases right but again this revelation would not happen in the absence of approval regulation because we've already had the announcement of phase 1 phase 2 and phase 3 trials this is all after all of that right so it can't just be could be a further scientific signal but it's a scientific signal from the regulator right and I think that's the key the other thing again is is to the extent that it really is about mechanism of action I'm not worried about like the whole world abandoning biotech I'd be much more concerned about saying look in this market like the FDA is being too tough or we've had this failure we should see basically high degree of class specific action and not non specific action it turns out that the FDA made the wrong very large estimates for when the FDA has an advisory committee and the advisory committee votes it down surprisingly right and that is consistent with the idea that it's not simply the FDA but also these scientific advisors giving a negative judgment on the drug right but again that's not the only place we observe a lot of these so if the FDA says no look we want another test or no we want a set of other things and again remember keep in mind all three phases of clinical trials have been completed for almost all of these at least to half right so it can't be just that you know a clinical trial previously when it's your right that there may be some revelation of scientific information still left but again that's only coming because we have this regulatory process so so here's the effects of one of these shocks all right and I'm just gonna generalize this to say all right let's just imagine one of these shocks is 10% drop in the sponsor stock price what happens to the hazard rate of abandonment for all other firms that is to say month by month by month what's the increased rate at which companies abandon their drugs given that 10% shock now one thing I do here is is t plus zero is the month of the shock so one of the things we do is actually we include some leads here and that's a test of two hypotheses one it's kind of what you might call a placebo test the idea that the shock should not be predicting something that they really can't predict which is abandonment ahead of time and it's comforting in this respect to know that these by the way these Reds are the parameter estimates these are 95 percent confidence intervals both individually and jointly these are zero okay the second is is this is a test of anticipate if in fact these things could in fact be hedged ahead of time you should see other companies adjusting their development strategies in the months before and again this is statistically zero right where one sees the effects is essentially beginning in the second month and continuing roughly if you want to sort of judge that is on the margin of statistical significant until about the six-month it takes time in other words for these to filter their way through firms and their decision processes to make judgments about this is by the way generic this is both therapeutic specific effects and non-therapeutic specific effects combined all right once you get out here there's just enough noise that there's really just not much going on if I run that linear probability model I talked about earlier okay so this is not this is a little less interprete belief you will this is what's the change in the probability of abandonment again we have to adjust the things it's for lack of a better term essentially the same results although a little bit less statistical significance we get these two T plus 2 and T plus 4 if you actually compare these two they have this essentially the same shape even though basically nothing going on early right around T plus 2 to T plus 4 arise whoops and and then down to where there's just a lot of noise all right which is comforting in the sense that basically the linear probability model relies heavily upon these fixed effects to generate a within subject treatment so it can't be any feature for the linear probability model excuse me it can't be any feature of the drug that's currently under development right it has to be only the shock that's generating this response all right and again notice that the lead values are all 0 so there's not anticipate here if I again just get rid of all the leads and everything past six months things bounce around a fair bit more all right but the average of this is quite positive if you will each 10% shock if I integrate over these distributions each 10% shot leads to about 4 to 6 drugs abandoned in the six months following okay we can't say that those drugs wouldn't have essentially become eventually become approved we can't say that they would have become useful treatment so they would have been marketed well all we can say is they have an increased probability of the firm's themselves pulling the plugs in response to that okay so now if we look within therapeutic category we look at this division chart these are the therapeutic categories we're going to use essentially this 14 and not 15 because this one is OTC over-the-counter drug products we're not looking at those so it could be skin and dental could be antiviral could be anesthetics it could be pulmonary things like that some of these names may be recognizable robert temple is one of the most influential people in the history of 20th century pharmaceuticals again he's got a now a kind of a top-level deputy commissioner post but at this point he was the head of one of these drug reviewing divisions this guy is often very controversial is often taken to task in the Wall Street Journal editorial pages as being sort of a drag on cancer treatments so some of these names are kind of very well-known if we look at the effect of the 10% shock and therapy targeted we get stuff that's very similar to what we had it bounces around a bit but very much similar to what we observed before the second thing we can do is say well what happens when we kind of break these events down by what was happening so let's just examine five categories and for those of you who do work in statistical text analysis or coding or content analysis this would be a great application of those kinds of methods basically look at what kind of decision this was and try to classify it but it could be a case where a company abandoned the drug on its own incited FDA regulation is a reason for doing so so we code that separately it could be an FDA request for more data it could be an advisory committee voting and saying no it could be the FDA saying we're not ready to make a decision on this yet okay each of these outside of the FDA saying we're just rejecting this all right doesn't seem to have an effect now remember one reason might not have an effect is because this is probably the easiest one to anticipate where the FDA on the deadline says we've made a decision up or down and that you know the firm is kind of communicating we're not creating getting great signals from the FDA so it's not surprising essentially that that's a zero technically it might be statistically significantly negative but I don't put much in it right the biggest effects are from when an advisory committee suggests no there's a bunch of reasons for that I would or hedge number one that's the first read on the FDA's thinking and outside committee which is going to advise the FDA after these phase trials right sometimes there's a public today now often there's a public report released by the FDA review or the FDA review team in advance but the period we're dealing with that report was often released at this meeting right so there's a whole bunch of things that are folded into here second this is a sort of a judgement not simply about what the FDA thinks but what a panel of sort of independent cardiologists who advise the FDA thinks so this gets in part to your question about you know to what extent is this a signal from science well again it's both but here again it's where we're letting the sort of advisers speak a little bit independently of the FDA as well right turns out that a fair degree happens just from the cases where the FDA says we're not ready to make a decision on this yet and it's tough to ferret out the reasons for that it could be that we'd like more data so we don't think that we think that it looks good but we'd like more proof about bigger sample size a smaller confidence interval or we're just you know we're not ready to make a decision yet so it could be you know the mailroom isn't working you need a plumbing repair on floor three something like that but that also also generates a higher degree of company abandonment and and other companies abandoning and citing the FDA as a reason or citing regulatory factors as a reason also leads to about a 4% increase in the hazard rate these by the way are summed across six months the I'm sorry seven the month of plus the following six months and here's what we do if we binarize the treatment right so this is where we've taken that stock price shock and we purge it all right and then we say all right we're gonna sign a 1 if it drops by more than 3% and 0 if it doesn't dry mean it doesn't drop by more than 3% and then we're gonna sum across 12 legs and essentially most of this effect is occurring within therapeutic categories right and that's a very large hazard ratio because that is being multiplied by month across firms many many times over so now if we take these as kind of our evidence we're talking about 30 40 50 drugs getting dropped after one of these events and not just a few but keep in mind that some of this is also occurring generically which is to say outside of therapeutic class so you can't ignore the fact that some people are making inferences not just about what the FDA oncology division is thinking but about the FDA writ large okay this is specifically coded as to say alright an oncology drug goes down what is the reaction of people in cardiology developing cardiology drugs or infectious diseases drugs and this is a case where we actually control for a few other things so what do plausibly abandoned drug projects look like well essentially we expect those with a shock and then what happens to periods afterwards all right which was one of the significant statistically significant parameter estimates that we have so we can't know whether these in fact were caused we just say it's consistent with the causal story these would be predicted to have a higher level of regular regulator induced abandonment okay so it turns out that over 95% of those abandoner in phase 3 which from an efficiency standpoint is bad news if you if you wanted these to be abandoned you'd like them to be abandoned early before all that capital and is sunk in right now I can't say whether over 95% of drugs that are abandoned are in Phase three because we don't have great data on where these things are and the further they go in the process the more likely they are to be reported at all so all I can say is for those drugs for which we have phase data phase 1 phase 2 phase 3 95% of these are in Phase two but that's highly highly selected because if you get to phase 3 in this database it's my more likely that the people who put this data gate base together are able to report that you're in Phase three what you can say I think though is that a fair number of these are in Phase three and are dumped right we can't say that 4,000 drugs word rumped because of regulatory factors right we can just say that among those that occur in these events right after the you know two four months after these a high number of those for which we know the phase seemed to be phase three all right and we have to sniff some more to kind of dig where that is most of these are again implicit abandonments and non explicit but if you look at the and I can send you the paper or you can even look at the previous slide you get very similar results as opposed to whether you focus on implicit abandonments or implicit abandonments alright so choosing one or another of those measures actually doesn't seem to affect much the results that you get from these estimations which is somewhat comforting so to conclude on this part well I think this is still speculative I mean one thing I'd like to be able to say is give you a harder estimate of well one when one of these things happens the following number of drugs are abandoned and they're abandoned in this phase and things like that there are some limits on the data which I think will prevent us from ever being able to do that in a fully satisfactory manner but one can do that it's also important to say that this is not an evaluation of what happens in response to regulation generally like the issuance of a new rule but the issuance of a regulatory decision right and that points I think to the difficulty of measuring the overall effects of a policy because regulations usually come in bundles right and regulatory decisions usually come in bundles so you say well let's evaluate the effect of this regulation on why well what part of the regulation are you picking up because the regulation is probably a statute right or a rule with seven different components and is it components – or component five so there's a lot of debate right now about what's the effect of the dodd-frank act on the financial realm well the dodd-frank act is you know Prudential regulation which is to say large banks its regulation of credit rating agencies its regulation of the home mortgage market make the new Consumer Financial Protection Bureau right it's 20 different things in fact really it's more like twenty thousand different things going on in that bill right and so you know assessing the effect of a piece you know of regulation writ large or a piece of regulatory statute religion is very hard because these things come in bundles and it's very difficult to disentangle one part of that component from the other and so the more you focus up the more you basically give up in terms of granularity the more you go in terms of granularity the less you're able to focus on regulation writ large I don't think this problem is fully escapable right I don't think it's possible to just say well there's a strategy out there that will allow us to speak about regulation writ large and also to have this kind of granular approach this is what I think at some level political scientists can teach to those who wish to evaluate policy policies come in bundles and it's hard to disentangle one part of the bundle from another right it's difficult also to draw policy conclusions again all I can say is that firms are more likely to pull the plug on these projects I cannot say right that these projects were of high value we might be able to follow later on in some of these therapeutic areas and say were there cost beneficial new products introduced what happened to morbidity mortality some public health measures in these areas where there were more surprise rejections we might be able to follow that but I haven't done it today and again the more you start to sort of take into account some of these therapeutic area specific measures the more you're beginning to sort of introduce other areas which can contaminate right there's no way of knowing essentially what the health effects would have been other words have these things gone to market or what the economic profitability would have been had these things gone to market that said this method does open the black box a little bit right we know that it's not simply the FDA rejecting a drug that might lead to more less innovation which is to say the FDA making a decision no on something that's sent to it but the effect that that is having on firms own decisions not to continue their own product development processes and not to seek approval for those projects later right it's also potentially generalizable in theory if you can find regulatory enforcement decisions in other domains focus them on firms compute what happens to those firms as to whether they're going up and down right and then I think this is the key can you get a large database with high granularity on what other firms in that domain are developing energy development projects right consumer financial or you know systemic financial innovation it's really that dependent variable kind of data that one needs to be able to evaluate the stock market data and in some cases the regulatory enforcement or decision data is always there what you really want is a high granularity database at the level of firm decision-making to be able to evaluate what happens with R&D so I'll conclude there and open it up to questions as other questions as you like I have a graduate student and I may end up joining her on that project or not but that's exactly one of the things where that's occurring I was wondering if you have data on who's in charge and yeah that's a great question so actually the woman who just asked a question has a copy of my book there thank you and I discuss one of the in the historical period an historical work that I do I just described this process of sub-delegation so in theory this power of veto is given to the secretary namely Kathleen Sebelius but in the 1960's and 1970's it kept on getting sub delegated to the fact that you've got career bureaucrats making these decisions now in a way that's almost never overturned by higher levels the only case recently and we talked about this at dinner last night where there's been an overturning was the plan B decision when Obama and Sebelius basically turned down the approval of Plan B for over a counter status but that's the exception that in some ways although I worry about the precedent that it might set for kind of overturning d'un yet it is possible and I did it a long time ago and I kind of gave up on it too you can get approval time data to net out the effect of different reviewers basically by like computing a fixed effect for each reviewer and then just to examine the fixed effects and I just never went very far with it but I've got all my data from this book online not all of it but a lot of it and if you went to it I could probably give you some others we basically coated the entire seeder employee directory from the 80s through the early 2000s so we have like 5,000 employees sitting in this database and you can see in many cases which one of them did the review with the review team composition was and you can net out the effect of a division director and things like that assumes of course that you're controlling for everything else that might be correlated with that so in theory that's possible but I never went so far with that is to do it in part because there's a lot of missing data on who made the decision in this case who made the decision it'd be easier to do in more recent years because the FDA is actually pretty good on the whole given the limits of the marketing so actually me two to two things I mean there is at some level kind of a continuous kind of information release about the way these things happen I'm not worried about that in terms of internal validity because again that gets priced in so I'm looking at you know what happens the day of what happens the day after that's another reason for focusing just on that you know one day shock but I think there's a more interesting process by which some of this gets so one of the other things that you could actually do by the way is look at what happens to other firm's stock values right after so I've looked at what other firms do with their development decisions you could look at other firm stock values the problem is that could be responding to a lot and in fact not least the regulatory decision itself like I'm a competitor in this market maybe it goes up because now there's space but more likely it probably goes down because they have to pass through the same gauntlet now hearings I'm not so concerned about but there's constant communication between the firm and and so you know what gets released to the marketplace things like that I mean so what we do know is that in theory the review team's deliberations are lockbox it's only at or just before a today an advisory committee that the review teams memos are put online there's often a lot of movement right there if we had more recent data we might be able to kind of exploit that the clinical trials are lockbox for a number of reasons one is blinding right so you know you can't you know inspect the data halfway through and say does this look like it's going well or look like it's not although if this idea for more Bayesian clinical trials takes off you might see more of that which could actually create some interesting problems with insider trading that I really hadn't thought about that and that's interesting but at least again that more traditional model now that's lockbox and that's in part FDA regs but it's also humans subjects and blinding rigs there's a lot of communication that goes on between these review teams and you know again the problem is is if you know if you're a company person and you're holding stock and you're privy to some of these the one advantage that the SEC has is it knows who's privy to that information right so it knows who has access to the database at the FDA and it knows all the people at the company and you will see people at these companies getting hauled into court and sometimes put in jail for having heard bad information and then going selling the stock or having heard good information ahead or probability and going and buying the stock or you know hedging one way or the other but a little bit of it does there is I mean it's a little bit more continuous that I'm stating here there are some huge discontinuities but there is a little more continuous knowledge everything how they say about academics so you is how Tony Carr the paper rejected by a journal and it gives us by by my urge to submit it to the same journal because there's no my expectation but in your case maybe the problem is there's only one general wherein the Toigo you don't write so you have no idea journals or something – so that is more psychological demand or would it be more like what the gentleman referred to is kind of revealing some kind of the underlying scientific promise of a certain lines of thinking so right what would be your take on that would be the actual well I actually think that not all this is purely rational expectations right but might in order for my story to work I don't this doesn't have to have full rationality it's the extent that people are kind of scared off by the FDA perhaps irrationally so that they should have continued on my story doesn't change because it's a it's a story about the effects of policy and I do think actually you know this mainly comes not from the quantitative research but years of looking at these industry trade journals like the pink sheet and other things like that there's a lot of fear in this industry because they recognize that they're sort of in in front of the all-powerful regulator and even though we like to tell stories about the pharmaceutical industry dominating the FDA that's number one a more recent development where the the pharmaceutical industry has had that kind of power and number two firm by firm these companies are still very afraid of the FDA and these drug reviewers and things like that so I think actually a lot of this is basically being scared off some of that fear may be irrational or inflated and some of it may be rational which is to say we think things have happened here it's hard to really nail the mechanism I think part of it is exactly what he's saying this is a revelation of science I actually don't think again that's inconsistent with the story because that revelation wouldn't be happening but for the regulatory process right in other words if you could just required everybody to go through three phases announce those phases and then go to market you wouldn't be seeing these effects because the three phase trials are already being priced in and once we've seen this right but I think a lot of this because precisely because it's happening both within therapeutic class and outside of therapeutic class is judgments about the FDA what I can't say here although I think I probably could with a little more confidence with some more data is whether this is the FDA raising the bar or more uncertainty about where the bar is and I think that's an important policy question my sense again just you know eyeballing the data is that some of both and I'll probably need to kind of do some auxilary tests to kind of you know do that but both of those are important questions you know you can make an argument that from a policy standpoint you might want to have higher bars or lower bars and certain points but it's always better off to maybe know where the bar is so have less uncertainty for the industry and for science and things like that although there's an argument that ambiguity can also serve purposes because it doesn't allow the rig the firm's to gain the system as much it keeps them kind of on their toes as well um but I do think it's being scared off whether it's by uncertainty or by the bar changing that that is that is the that is probably the the mechanism here so here's again the problem this gets – yawns nice point about there being one journal editor right one reason that the FDA is so powerful is because it controls access to the most powerful the most profitable pharmaceutical market in the world so yes if you want you can go introduce your market your drug to the European market but it's going to be price controlled it's gonna be in a company that in a country that's not as rich and it's actually not ageing as fast as ours right so we have high pharmaceutical consumption basically zero price controls at the margins with a couple and maybe with Medicare Part D in the future we will but what makes the FDA so powerful in this world is precisely the fact that it's a stringent regulator in a world where price regulation is not stringent so gatekeeping power in other words is directly proportional the amount of gatekeeping power to the prize that you're keeping aspirants from right and the fda doesn't control the fact that there aren't pricing regulations in the US and it doesn't control the political economy of the United States but it's gatekeeping power benefits at some level from these other factors I think coming back to this point about in Canada one thing are struggling with boys what do you think you've convinced that these changes for exotics that you don't they were in like a random shot and we were big change so that sometimes I don't know whether they were just transient or and was this having absolutely and rejecting heard from or was it's no change in the FDA stands about death threshold and so I can't say that yet we could write because I could say alright let's this followed by later decisions I don't know how to those legal decisions I'm more complicated because I said because I've been dodging ad there now you know there's a higher threshold you don't take drugs to the FDA which I think of failure and once you begin to analyze the process in this way being at least open the door to answering some of those questions but I agree I haven't done it yet I mean essentially what we want to do is Trace not only the decisions as we do my shops bus series of the decisions you know cells and say is there a pattern here and some level that's kind of descriptive and bare-bones but I think you kind of need to do it step away from the internal validity church or a minute and then you know kind of focus more on kind of descriptive features and that again allows us to go a little more from regulatory decisions to regulation at large [Applause]

24 thoughts on “The FDA and the Pharmaceutical Industry

  1. The drug industry should be made up of a number of small industries that could compete with each other rather than a few large industries as it is today.

  2. FDA is scape goat to kill all long term patients surgeries like fankenstein and promises that change every week. Long term patients know Truth of how we have lived all the changes. Check out Greensboro NC, Theresa mitchem. I have 3 feet of paper files, not just what they decide to write on web. You don't know it until you see your grandchildren crying cause doctors, FDA or government just want to act like we are healthy as can be. I am speaking for all the patients told to be healthy, when years of paid doctors have cut them all up, because of health. You need to Grandfather us in cause we have served the time. PS please don't let anything hurt you 🌷✝️

  3. The pharmaceutical companies create customers not cures Remember Remember the tunksgee project the study of untreated syphilis on african Americans back in the past?

  4. The top people in the FDA need to go to prison for 200 years with no parole because of colluding with the drug companies and mass murder.

  5. Great information sharing….keep up the good work going….also get medicine and pharmaceutical products in wholesale at the prime distributor in USA

  6. People do not understand that pharma works on creating an ideal drug, a safe, effective and selective substance which can cause a therapeutic benefit with little to no side effects, however sometimes efficacy may be more important than safety…
    people keep blaming pharma about producing opioid medications which are highly effective with their own set of side effects. This is the reason why medication have warning labels on them, they are there for a reason to make the public realise that there is a chance of adverse effects and addiction to the drugs they are taking, people don't understand the medication they are taking, get addicted to it and than blame the pharma industry….
    Yes that drug isn't completely safe but it treats your pain and illness, and that maybe due to low knowledge related to the drug and that maybe pharma's fault, but at the same time people are also responsible for their actions….

  7. FYI…over 90 Integrative, Holistic, Chiropractic doctors murdered in the past two years. They spoke out against vaccines and advocated natural treatment protocols for Cancer. Where is mainstream media & ongoing police investigations ??

  8. The FDA is needed, for every nation requires strict regulations on food and drug, consumed by not only local but importing countries. We (health gurus) would greatly appreciate laws that allow us to use our NATURAL DRUGS which has no side effects for our maladies. Many/ some of my physicians refuse to come to terms with patients like me, who prefer HERBAL ALTERNATIVES. It will be very pleasing to MANY if BIG PHARMA & FDA can kindly consider our many affirmative testimonials providing proof that ALL HERBAL TREATMENTS are effective. Our physicians must respect our constitutional rights to allow treatment by the same without ridicule or discouragement.
    I would love to suggest BIG PHARMA not to be jealous of the BILLION DOLLAR HERBAL INDUSTRY. Why not create new farms and labs that produces ALL HERBS yourselves? YOU CAN TAP INTO THAT MARKET ALSO, INCLUDING THE FDA (government). YOU CAN ALSO DEVELOP ORGANIC FARMS AND NATURAL WATERS. Great marketing and investment skills, NOTHING IS HINDERING YOU ALL, POWERFUL PRIVATE AND PUBLIC AGENCIES.
    Christians are not the only consumers of natural foods, toiletries, water, herbs, household detergents etc. Many health gurus, healthy people, fitness and personal trainers etc, even (I hate to inform you) White Magic people USES MANY HERBS AND ALSO OTHER RELIGIONS.
    We ALL know we must ALL expire at some time in our lives, some younger, some older but ALL MAN MUST DIE. Is it not BEST to die at your destine time as appointed by God, than being a burden on your love ones and medical staff? Being re- admitted multiple times due to taking poisonous MEDICINES, FOOD, WATER etc. Only scoffers mock saying SOMETHING MUST KILL YOU, even HEALTHY VEGAN DIES! We ALL are aware of so many cases of the same, that is besides the point. Our bodies are Gods temple, we are commanded to keep it holy. How can we ENJOY LIFE, A GIFT, being ill? Be it mental, emotional, sexual, physical or Spiritual?
    Take care of you bodies for health sake and to ENJOY LIFE ABUNDANTLY, if you are not a HEALTH "freak" . Why not grow old then die, gracefully, without NEEDLESS PAIN BEARING? Please give us more rights! 8 remedies are :

    Trust In Divine Power

    Then herbs, surgery if needed, dentist if needed (natural fillings). Share this with the world search the KJV Bible and EGW books / Ministry Of Healing/ Counsel Of Diet Of And Foods. etc. Converse with E.G. White Estates, U.S.A. We sincerely hope there would be a decline in DEATH BY MEDICINE. Thank you Gods!

  9. Representative exclusive ITT and CME internationally make claims for intercellular modules treatment regarding efficacy Regulatory practices academicians have narrowed accessibility and limited wellness. WHO world health organization has a responsibility to inform education and discipline it patients and constituents on WELLNESS . Practitioners should have ancillary and Auxcillary solutions . When do we stop treating the disorder and cure the system streams?

  10. Not one person in the FDA/Pharma Mafia gives a damn about this USC Price BS.  Public apathy is the real enemy. The US is infected with Liberals, DemoRATS, Socialists, atheists, faggots and dipshits. It is way past time for another Civil War.

  11. The FDA/Pharma was and remains an EVIL government Mafia entity. Any adult employed by them is a traitor. they play on public gullible/naïve and  apathy to exist.  They are worse then the Mafia and Gestapo.

  12. The Supreme Court isn't any betterThey ruled that if we are harmed by generic drugs we cannot sue for damages. They never gave any advice on who to sue or what to do. Insurance companies must be jumping for joy. They do not have to pay anything for damages if we take generic drugs. I took a generic for pneumonia Levofloxicin this is a floriquinilone antibiotic. I was not told that I possibly could suffer an aortic dissection. Almost died! there are class action lawsuits for the same drug made by the brand name manufacturer Leviquin. I took the exact same formula pill, yet I HAVE NO LEGAL RIGHTS! tried over 60+ lawyers all I got was this advice "GET SOMEONE TO REPRESENT YOU BEFORE THE STATUTE OF LIMITATIONS RUNS OUT" IMPOSSIBLE!!! After 60+ tries no one will represent me.

  13. The FDA needs to be exposed. The American citizens need to know how corrupt they are. Why are people not getting this info out to the public is beyond me. Come on! people are dying and getting sick from all these "approved" drugs and "approved" substances that are in our drugs and food.

  14. Clinical trials are worthless.
    In Woods , bears don't have MD's, clinical trials and they are healthier than humans. Parasites !

  15. EVER HEAR OF THE UNITED STATES COURT OF FEDERAL CLAIMS, THE NATIONAL VACCINE INJURY COMPENSATION PROGRAM???  GO HAVE A LOOK AT THE LINK FOR YOURSELF BEFORE YOU DECIDE, MOST PEOPLE HAVE NO IDEA IT EVEN EXISTS!!!! The National Vaccine Injury Compensation Program ("Vaccine Program") comprises Part 2 of the National Childhood Vaccine Injury Act of 1986 ("Vaccine Act"). See Pub. L. No. 99-660, 100 Stat. 3755 (1986) (codified as amended at 42 U.S.C. §§ 300aa-1 to -34). The Vaccine Act became effective October 1, 1988. It establishes the Vaccine Program as a no-fault compensation program whereby petitions for monetary compensation may be brought by or on behalf of persons allegedly suffering injury or death as a result of the administration of certain compulsory childhood vaccines. Congress intended that the Vaccine Program provide individuals a swift, flexible, and less adversarial alternative to the often costly and lengthy civil arena of traditional tort litigation
    Show less

  16. Most drugs end up not working cause of too many side effects. But there are a few that are ok but not many. God has created herbs that most drugs are made from anyway. 

Leave a Reply

Your email address will not be published. Required fields are marked *