Intrigued, I looked closer. To their credit, the BLS publishes their entire methodology online. All you have to do is wade through explanations of statistical number crunching as described by Washington bureaucrats. What I found suggests that Crudele may have been understating the problem. When you actually reproduce the BLS methodology described in the BLS Handbook of Methods (Chapter 2), you arrive at the conclusion that fully 85% of the new jobs claimed to have been created since March 2003 are imaginary.
To arrive at a monthly estimate of nonfarm payrolls, the BLS creates a benchmark universe of jobs through compiling Unemployment Insurance records from all 50 states and the District of Columbia. This benchmark is updated each year, about eleven months after the fact. The latest benchmark data is March 2003, which was compiled in February 2004. Then, each month the BLS takes a random sample of the universe and counts the jobs found within. They then compare that figure with the corresponding figure from the prior month's sample. So if the latest month's sample found 26,000 jobs, and last month's found 25,000, then the conclusion is made that the job market grew 4%. They then multiply that percentage growth against last month's estimate of total jobs. But that's not where it stops. They then arbitrarily add a figure for jobs created by new businesses they imagine were created, based on the number of businesses went dead that month (which are signified by the number of businesses in the sample that either reported 0 employees or didn't report at all.) The assumption here is that a dead business in the sample automatically means another business was created that month that hadn't gotten around to report to the state unemployment insurance agencies. Of course this doesn't account for all those businesses who laid off their employees because their jobs were outsourced to India, but I'm sure that's not a problem. Right.
The insidious thing about this is that these imaginary numbers appear to be cumulative. That is, if BLS imagined that 1,000 jobs were created in one month by businesses they can't see, then that 1,000 gets added to next month's total as well. Here's the formula:
(Last month's Total Jobs X Growth Rate of Sample) + New Jobs Imagined By BLS = This Month's Total Jobs
So this month's imaginary figure gets rolled into next month's figure, and next month's etc.
Applying this method to the actual nonfarm payroll data, we find that 1,104,000 of the 1,303,000 new jobs reported by the Department of Labor since March 2003 are basically made up out of thin air. Here's how it translates graphically:
When this birth/death model was created in 1998, it was expected that it would change the results to a modest "small and stable" degree. Indeed for the 11 months from March 2003 through January 2004, a modest 37,000 average jobs per month were added by this technique. Something happened though after that. Over the last four months (February through May 2004) this modeling technique padded the total job growth by over 730,000 jobs, about 183,000 per month.
In February 2005, there will be a new benchmark calculated from universal UI data that will reconcile these figures to the real world at least through March 2004. Most likely, what will be found is that job growth came in somewhere in between the 1.3 million announced and the 199,000 jobs actually detected. This is because there should be SOME new business creations adding jobs. However, how many of those businesses reported zero jobs to BLS did so because they outsourced to India instead of creating a vacuum where a new company grew domestically? When given other well known measures of employment showing no movement at all through this period, we are likely to be surprised how bad things really have been all the while our leaders were telling us how lucky we are.
(Thanks to comments shown in the comments page for this post, the numbers are slightly revised. See comments for full discussion.)
22 comments:
Without doing the math, and assuming your calculations are on the money, it looks like another episode of "padding" is going on (where do we start? deficit, etc). Only this time its exponential.
I always wondered about the quality of new jobs being reported, and keeping in mind that number of people who haven't found jobs, and so no longer counter new growth, but did not think of the "out sourcing" sink hole as you describe.
Like the lie about decreaes in "terra attacks" this is just another piece of the Bush criminal puzzle.
Hitler knew, as most facists do, that fooling the people is the easiest thing in the world to do. Thank God for the interent, it, and not the First Amendment, may be in the long run what saves us.
The world will be a better place once the criminal thugs of the Bush cartel are long removed from power -- but they are posed to steal the next election (voting machines, another Reichstag fire, Osama?) and if so; I am afraid the next generation may have to endure the ugliness that has eminated from the Reagan-Bush capitilists. TV, Oil and Propaganda are the hallmarks now of an America with an very uncertain future.
Your analysis has two immediately apparent flaws:
1. You are using the seasonally adjusted non-farm payroll series as the base for your analysis of the percentage of jobs added by the Birth/death model. The Birth/death numbers are not seasonally adjusted therefore you need to compare them to the non-seasonally adjusted series. This will cut the contribution of jobs attributed to Birth/death model to approximately 40% for the 2004 calendar year.
2. You oversimplify the bith/death model. It doesn't just calculate jobs for business births based on business deaths. It also includes a times series model to account for the number of new business jobs that historically have not been predicted by business deaths.
Here is a paper describing the model in detail.
I haven't had the time to examine your claim that the Birth/death methodology leads to accumulating errors through time. I doubt that BLS would make such a glaring methodological error.
I commend you for taking a shot at this important topic but I think you need to take a second look. When put numbers in the proper seasonal perspective, the birth/death numbers still do seem a little high. But the birth death model has been carefully constructed and tested on a sector basis in the nineties so I tend to suspect the numbers are not upwardly biased.
FTM
To FTM:
Thanks for your comments. I will indeed take a look at the non-seasonally adjusted numbers as you suggest. The BLS database is down for revision at the moment, so we'll see what comes up when it's back up. I did take a look at the latest release which shows that in the non-seasonally adjusted series there were 1.4 million jobs added between May-2003 and May-2004 versus 1.0 million in the seasonally adjusted series, so you clearly have a point.
On your second point you are correct I did not point out the additional jobs added as a result of applying the ARIMA model, as it was not important to get across the major point to a largely lay audience. Both procedures have what may turn out to be an underlying flaw. As you point out the birth/death model was vetted rigorously in the 90's. However, statistics are great at measuring correlation but completely lousy at detecting causation. The behavior of corporate birth/deaths and their effect on job creation may have little relationship in recessionary/recovery periods versus boom/bubble periods.
Throughout history you can find correlations between population birth and death rates. And you can construct birth/death models around them that are predictive a great deal of the time. However, such a birth death model applied, say at the time of the medieval Black Plague, would have predicted an incredible population explosion 'imputed' by the epidemic deaths.
The fact that at this point the birth/death model is producing results far from the "small and stable" numbers that the producers of the model expected shows that there may be something very amiss with the whole procedure, and that something fundamentally different is going on in the American workplace. What is it? We won't know as long as the BLS simply announces good news, content in the realization that few reporters will read their own caveats and cautions at the bottom of the press release.
Thanks again for comments and constructive criticism. They won't be ignored.
-RT
Glad to hear your going to follow-up. Hopefully the data will be available soon. When I looked at the data, my back of the envelope calculation was that the Birth/Death model contributed about 40% of the jobs in both the portion of 2003 and the portion of 2004 for which data was available.
Forty percent seems like a lot to be getting from new entities and I have some questions about what qualifies as a new entity. What might be happening is that a lot of the jobs BLS is calling new are not really new but are old jobs for which the reporting entity has changed. For instance when corp x sells a subisidary to corp y, the jobs turn up classified as new. This would also help to explain why jobs from business deaths are so predictive of jobs from births. I haven't seen this explanation from BLS but then I haven't yet found any BLS literature about tracking jobs when entities change reporting identifiers due to reorganization.
I'm going to be off-line for a week but I'll check-in then to see if you've got an update.
Thanks for your constructive response
FTM
Well FTM I finished taking a look. Here's my response:
FTM raised two objections to my findings. First, he pointed out that the net birth/death model included two parts, the second part being an ARIMA time series adjustment that adds yet more jobs though to be missed in the first round of new job imputation from known business deaths. Let's acknowledge right away that he's absolutely spot on in my failure to describe this second step adequately. However, I do not think this criticism changes my analysis at all, since the numbers I've used to describe the effect of the birth/death model are those reported by the Labor Department as being the product of both steps.
FTM raised a second objection that was harder for me to dispatch. Keep in mind that my knowledge of and use of statistics is in practical applications. I'm not an academic statistician and am not likely to ever be one. FTM points out that the BLS uses seasonally-adjusted and non-seasonally adjusted data. FTM argued that the birth-death model numbers were non-seasonally adjusted so I should be applying them to the non-seasonally adjusted series of total jobs. By doing so, he posits, the effects of the birth-death model are much less than I had claimed.
I first conducted the same exercise as had produced my first set of data, but this time using unseasonally adjusted data. Doing so I was able to faithfully replicate FTM's estimate of the 40% difference in jobs with and without jobs "imputed" by the birth/death model.
The problem though is that virtually all of the difference between the unseasonally adjusted and the seasonally adjusted numbers have nothing to do with the circumstances surrounding the birth death model. More imoprtantly, the unseasonally adjusted numbers are not the ones presented as quotable news by the BLS. So an additional step is necessary to see the effect of the "imputed jobs" on the seasonally adjusted numbers. So here's what I did:
I don't have time to download the BLS's ARIMA software and replicate their seasonal adjustments. (I have a life.) However, we know the net effect of those adjustments for each month since March 2003 because we can see the before and after numbers of each seasonal adjustment. So, we take our two sets of unseasonally adjusted data, one with the 'imputed' jobs added in, the other without, and apply those same seasonal adjustments to both sets and then scan the difference between the two, That difference will be the net result of adding those imaginary 'imputed' jobs to the seasonally adjusted database.
The result I'll post graphically on the main page, but in a nutshell the deal is this, by properly taking into account FTM's objection about properly handling the whole seasonal adjustment issue, instead of 88% of the 1.3 million jobs said to have been created since March '03, the more correct figure is 85%. By straining the birth/death model through the seasonal adjustment, it changes the number of jobs actually detected through sampling (ie: real) to just under 200,000. 1.1 million jobs are imputed to exist through statistical methods the veracity of which can't be checked for months.
My thanks to FTM for his points, allowing me to clean up the numbers a bit. I will edit and re-publish that post with the new graph and new numbers.
In the article "The Education of David Stockman" William Greider relates how the architect of Reaganomics came into office and immediately crunched the numbers on the tax cuts they wanted to give on "An OMB computer, programmed as a model of the nation's economin behavior". The numbers that came out were so onerous that Stockman declared them "mud", "absurdities" and said they show that "the world doesn't work". So Stockman and his team made new models and called them reality.
In fact though, the results of Reaganomics were so bad that within years they ended up raising taxes again, first on the middle class with the payroll tax increases and later the corporate AMT was put on corporations. So it seems the lesson from that is that the Bush administration should not be making up new models, and are probably doing so just to fudge numbers.
Before someone led me to your great analysis, I was disturbed by the 1.2 million new jobs claim. I wanted to see where that was. I'm a simple person so I simply went to the BLS Employment Situation Summary put out after the end of May. Looking in the top chart right below the number for the "civilian labor force" we see "employment". For May that number is 138,772,000. Now if we look up the "Employment" number for December that was released in early January (archived at ftp://ftp.bls.gov/pub/news.release/History/empsit.02062004.news ) we see 138,479,000. Subtracting Dec from May we get 293,000 more people employed since the start of the year.
Now I know that the numbers that the Bushies are looking at are down in the non farm employment, but they also have to be incorporated in the total. So if non-farm employment has risen by 1.2 million yet total employment has only risen by 293k then there has been a huge drop in farm employment in the last few months. Even though mid winter has some action in CA and Florida and possibly other Southern states we shouldn't see such a huge drop in "farm" employment numbers from December.
I'm not an economist, but it seems to me your analysis is spot on while the commenter's is not.
It seems to me that the Orwellian underpinning of the company death/job creation model is that if more companies die, then more new companies will take up the slack and more jobs will be created. However, applied to 1929-1933, this model would seem to imply that the Hoover years should have produced the greatest per capita job creation in U.S. history--when, in fact, this era produced the greatest collapse of U.S. jobs ever. So my question is, is it not possible that the massive "dying-off" of existing companies that underlies this model portends, not an increasing recovery, but, rather, the foreshadowing of the major U.S. depression we've all been worried about, but didn't really want to talk about?
You've pointed out another direction in which we see the problem with the "birth/death" model. The BLS actually has two datasets they use - the household survey and the establishment survey. The former is just what it sounds like, a survey of households from which they extrapolate info. The establishment survey measures jobs reported by establishments (strangely enough). My reading of the BLS material suggests that the birth/death model is only applied to the establishment survey, not the household survey.
On the other hand the household survey has it's own weaknesses. For example, a guy laid off from his job who then sets up a basement office and calls himself a consultant is considered to have lost one job and found another one. So the higher numbers found in the household survey would have to be considered a least a little bit wishful thinking.
Best, RT
Sorry to take so long to respond to your update.
It looks like we’re going to have to agree to disagree here. Rather than converging on common position, our positions appear rather fixed.
Re: the Birth/Death model residual correction component
You state the net Birth/Death component “adds yet more jobs thought to be missed in the first round of new job imputation from known business deaths”. This is a misunderstanding of the model. The net Birth/Death correction component can be either positive or negative and thus either add or subtract jobs. In a sector where many firms are exiting, (telecommunications for instance) the component would be negative. You are correct that this issue has no bearing on your numerical analysis but it does lead readers to mistakenly believe the BLS model is ill-conceived.
Re: How seasonal factors impact the Birth/Death model’s contribution to the monthly non-farm employment numbers.
We agree that BD model contributes approximately 40% of the new jobs if the numbers are not adjusted for seasonality. To get from the unadjusted 40% figure to a seasonally adjusted figure, we need to adjust the numerator (the number of new jobs attributed to the B/D model) for seasonality. There is no dispute about the denominator as BLS supplies the seasonally adjusted total employment figure. The numerator adjustment is at issue.
The first thing to note is that the short series of BD numbers available are highly correlated with the seasonal variation in the overall employment series. There is clearly a seasonal component to the BD numbers and a need for seasonal adjustment.
The method you propose for seasonal adjustment of the numerator is equivalent to assuming there is no seasonality to the BD model. Therefore it is not surprising that you find little change from your prior analysis. Because seasonal adjustments merely shift jobs in time, it is impossible in an analysis spanning an entire calendar year for the seasonally adjusted contribution of BD model to differ from the unadjusted contribution by the magnitude you suggest.
There is a simple way to calculate an approximate seasonally adjusted numerator. Think of the series of seasonal adjustments as a bank account with overdraft protection. At the end of every month, a deposit or withdrawal is made and a cumulative balance kept through time.
In March 2003, the data set starts with a seasonal adjustment balance — the difference between the unadjusted and seasonally adjusted total non-farm employment series -- of negative 773K jobs and the year ends with a positive balance of 827K jobs. To calculate the 2003 portion of the seasonal adjustment job account due the BD model, an assumption is required about what portion of the beginning and ending balance was due to the BD model. Since 40% of all jobs are due to the BD model over the span of the data set, forty percent is the obvious assumption. This results in a beginning balance of negative 309K and an ending positive balance of 331K. We also know that the BD model contributed deposits of 695K jobs over 2003. Given the beginning and ending balances and net deposits, 2003 withdrawals -- number of jobs contributed to the seasonally adjusted employment by the BD model -- turn out to be 55K or 48% of the total seasonally adjusted employment. For 2004, the number turns out to be 39% of the total seasonally adjusted employment. To check this methodology makes sense, we can also calculate that over the entire sample period the BD model does in fact contribute 40% of the seasonally adjusted employment.
Based on this analysis the contribution of the BD model to the total seasonally adjusted non-farm employment numbers in 2004 is actually less in percentage terms than in 2003 – 39% versus 48%. It suggests that the recent influence of the BD model is not disproportionate and the criticism of the Birth/Death model likely wide of the mark.
Thanks for your detailed response.
FTM
To FTM: Thanks for your comments. I appreciate the time you've taken on this. On one level, the fact that the only meaningful debate here is whether 85% versus 40% of annnounced new jobs are fiction pretty much makes the whole point - a substantial amount of the advertised jobs recovery is a mere statistical fiction absent real data.
Your point that the net birth/death model sometime reports negative numbers is well taken. However, since it went 'live' in March 2003, it's done so only twice, and at no time since January 2004. If my wording had to effect of portraying a poorly conceived model, well then so be it. I think it IS a poorly conceived model as it is being used today. To their credit, the BLS sees it the same way and discloses "The most significant potential drawback to this or any model-based approach is that time series modeling assumes a predictable continuation of historical patterns and relationships and therefore is likely to have some difficulty producing reliable estimates at economic turning points or during periods when there are sudden changes in trend. BLS will continue researching alternative model-based techniques for the net birth/death component; it is likely to remain as the most problematic part of the estimation process."
As to your math, I believe I have to stick with my figures. BLS procedures call for the addition of jobs imputed by the birth/death model to the jobs actually found in the sample survey on a cellular level. Then, all industry cells are added up to a total sum. Then and only then are the ARIMA seasonal adjustment calculations run. So the numbers of the birth death model are given the exact same seasonal adjustment as the rest of the survey at the same time.
Here's what I did: Let's say one month the non-seasonally adjusted number were 100 million, and the seasonally adjusted number was 99 million. The BLS also told us that the birth/death model had added 100,000 to that month. At your recommendation that I consider the net birth/death model numbers as unseasonally adjusted, I applied that 100,000 to find that the unseasonally adjusted jobs actually detected by survey was 99,900,000 that month. Since the seasonal adjustment that took place that month was a 1% reduction in total jobs, I applied that same adjustment to the jobs actually found by survey and arrive at 98,901,000 or 99% of the unseasonal number. From this I find that using seasonal numbers 99,000 jobs listed were put there due to the birth/death model.
While I am using a shortcut in that I'm not running the exact seasonal adjustment software to arrive at the numbers, I believe that I'm faithfully replicating the order of number crunching described by the BLS.
Best,
Richard
To RT: I’ve invested the time here because I wanted to try to understand the numbers and a difference of opinion is always a good motivator to keep sifting the minutiae. As you would expect, I’ve got some comments on your latest post.
>RT
Thanks for your comments. I appreciate the time you've taken on this. On one level, the fact that the only meaningful debate here is whether 85% versus 40% of announced new jobs are fiction pretty much makes the whole point - a substantial amount of the advertised jobs recovery is a mere statistical fiction absent real data.
>FTM
The BD jobs are real. There is just a lag in reporting them. This is a fundamental point. It’s fine to question the size or methodology of the adjustment but to claim that the adjustment is fictional is really extreme. It brings to mind the winger opposition to the statistical adjustment of census numbers in low-income neighborhoods. Certain numbers cannot be reported in a timely way. It would be incredibly foolish not to estimate them.
The seemingly high percentage of 40% is likely due to reorganizations. When firms move form state to state or establishments are sold, the identifiers change and existing jobs turn up in the Birth/Death number. However, this misclassification does not skew total employment, since the job is no longer counted under the old identifier.
The 85% versus 40% highlights a fundamental logical problem with your analysis. Seasonal adjustments can only significantly change the average numbers over short windows ( half-years, quarters, months) . When an analysis spans a year or more the non-seasonal and seasonal averages have to converge. If the BLS seasonal adjustments were as bad as your analysis implies, BLS statisticians would be professionally crucified.
>RT
Your point that the net birth/death model sometime reports negative numbers is well taken. However, since it went 'live' in March 2003, it's done so only twice, and at no time since January 2004.
>FTM
This is not correct. Bls does not release the net birth/death adjustment numbers. It only releases the total birth/death numbers. The total birth/death number consists of imputed jobs due to business deaths plus the net birth/death adjustment. Therefore, the net birth/death numbers could have been negative in many months.
>RT
If my wording had to effect of portraying a poorly conceived model,
well then so be it. I think it IS a poorly conceived model as it is being used today. To their credit, the BLS sees it the same way and discloses "The most significant potential drawback to this or any model-based approach is that time series modeling assumes a predictable continuation of historical patterns and relationships and therefore is likely to have some difficulty producing reliable estimates at economic turning points or during periods when there are sudden changes in trend. BLS will continue researching alternative model-based techniques for the net birth/death component; it is likely to remain as the most problematic part of the estimation process."
>FTM
When employment is starting to grow more rapidly, ARIMA methods will tend to underestimate, not overestimate, new employment. In order for ARIMA methods to overestimate new employment, employment growth would have to be slowing – I don’t know of anyone making this claim. Employment growth is weak but there is no evidence I’m aware of that it is deteriorating. Thus the weakness inherent in ARIMA analysis does not support the view that new employment is overestimated.
>RT
As to your math, I believe I have to stick with my figures. BLS procedures call for the addition of jobs imputed by the birth/death model to the jobs actually found in the sample survey on a cellular level. Then, all industry cells are added up to a total sum. Then and only then are the ARIMA seasonal adjustment calculations run. So the numbers of the birth death model are given the exact same seasonal adjustment as the rest of the survey at the same time.
Here's what I did: Let's say one month the non-seasonally adjusted number were 100 million, and the seasonally adjusted number was 99 million. The BLS also told us that the birth/death model had added 100,000 to that month. At your recommendation that I consider the net birth/death model numbers as unseasonally adjusted, I applied that 100,000 to find that the unseasonally adjusted jobs actually detected by survey was 99,900,000 that month. Since the seasonal adjustment that took place that month was a 1% reduction in total jobs, I applied that same adjustment to the jobs actually found by survey and arrive at 98,901,000 or 99% of the unseasonal number.
>FTM
It is logically flawed to apply the same seasonal adjustments to both series. This assumption precludes finding any difference between seasonally and non-seasonally adjusted BD jobs. Your method is an identity. You can see this by dividing total BD jobs by total seasonally adjusted jobs. You get the same 85% without all the calculations. There is no difference between your method and simply assuming there is no seasonality to the BD numbers.
>RT
While I am using a shortcut in that I'm not running the exact seasonal adjustment software to arrive at the numbers, I believe that I'm faithfully replicating the order of number crunching described by the BLS.
>FTM
Shortcuts are a necessity. With this short a publicly available data series, formal analysis is impossible. But shortcuts still have to take you somewhere new.
FTM
Here's my comment to your latest points FTM:
FTM: The BD jobs are real. There is just a lag in reporting them. This is a fundamental point. It's fine to question the size or methodology of the adjustment but to claim that the adjustment is fictional is really extreme. It brings to mind the winger opposition to the statistical adjustment of census numbers in low-income neighborhoods. Certain numbers cannot be reported in a timely way. It would be incredibly foolish not to estimate them.
RT: I think we're just caught up in semantics here. A job actually counted in a survey is a real datum. One not observed but merely estimated is not a real datum. In this case I call into question the basic premise of the estimation method. Estimation methods should be based on real world correlations brought about by causitive factors. Accidental correlations can vanish at any point in a time series. This model assumes the patterns of business birth and death are similar in both boom and bust of the business cycle. I say this because the model was backtested in boomtimes and worked great, then applied to work in the bust period of this last business cycle. I don't think assuming that business deaths and birth behavior is the same during boom and bust is a legitimate assumption to make. Further, with increasing globalization, the nature of employment and hiring is fundamentally changing as we speak. I don't think this model captures that at all. How can one assume that a business death means a business birth steps in, in the same region in the same industry in an era when a business death could well mean that the enterprises' function in the economy is now performed in India, or Mexico, or by a computer program run out of the company's main office? The fact that the birth/death model is suddenly detecting four times the imputed jobs than were detected by the same model (on paper) in 2002 or 2001 suggests that something has gone seriously awry. Consider: in "Accounting For Business Births And Deaths In CES: Bias Vs. Net Birth/Death Modeling", BLS reports that in March, 2001 through March 2002, 154,000 net jobs were imputed by the model. For the following year, 156,000 jobs were imputed. Suddenly, when the Birth/Death Model went live in March 2003, the subsequent year saw jobs imputed by this method jump to 642,000, a fourfold increase. In the two months since March 2004, we've seen an additional 435,000 jobs imputed. A rate that, if extended to the coming year, will be another fourfold increase! )
FTM: The 85% versus 40% highlights a fundamental logical problem with your analysis. Seasonal adjustments can only significantly change the average numbers over short windows ( half-years, quarters, months) . When an analysis spans a year or more the non-seasonal and seasonal averages have to converge. If the BLS seasonal adjustments were as bad as your analysis implies, BLS statisticians would be professionally crucified.
RT: Not at all. Your 40% number comes from the time period of January through May 2004. My 85% number comes from March 2003 through May 2004. Thus they aren't directly comparable. Further, it's a fact that the nonseasonally adjusted numbers from March 2003 through May 2004 are widely different than the seasonally adjusted changes. Using unseasonally adjusted numbers one comes up with over twice as many new jobs as there were reported by the seasonally adjusted series (2.7 million versus 1.3 million) in the 3/03-5/04 period. This is the reason why our numbers are so far apart.
FTM: This is not correct. Bls does not release the net birth/death adjustment numbers. It only releases the total birth/death numbers. The total birth/death number consists of imputed jobs due to business deaths plus the net birth/death adjustment. Therefore, the net birth/death numbers could have been negative in many months.
RT: We got caught up in words here again. The BLS reports their total product of both steps as "Net Birth/Death Adjustment". It is the effect of the total product of their two-step process that we are examining here.
FTM: When employment is starting to grow more rapidly, ARIMA methods will tend to underestimate, not overestimate, new employment. In order for ARIMA methods to overestimate new employment, employment growth would have to be slowing – I don't know of anyone making this claim. Employment growth is weak but there is no evidence I'm aware of that it is deteriorating. Thus the weakness inherent in ARIMA analysis does not support the view that new employment is overestimated.
RT: This is dangerously close to a tautology. Few are claiming slowing employment growth merely because very few are like you or I torturing these numbers to death to find truth and are instead taking the huge numbers out of BLS at face value. Frankly, if the BLS are "imputing" over 700,000 new jobs by virtue of what they see in business deaths, what that suggests by definition is that actual sample surveys revealed to them that about 700,000 jobs were eliminated due to business deaths in the last four months.
FTM: It is logically flawed to apply the same seasonal adjustments to both series. This assumption precludes finding any difference between seasonally and non-seasonally adjusted BD jobs. Your method is an identity. You can see this by dividing total BD jobs by total seasonally adjusted jobs. You get the same 85% without all the calculations. There is no difference between your method and simply assuming there is no seasonality to the BD numbers.
RT: Perhaps it is logically flawed to apply the same seasonal adjustment to "both series". However, I find no evidence on the BLS site that they did anything different and much evidence that indicates that this is exactly what they did. First, looking at the BLS's explanation of seasonal adjustment of the employment statistics, we find no discussion of difference of treatment of birth/death model generated numbers at all. Just a general discussion of taking totals found at the three digit NAICS level and applying the seasonal adjustment program on them. The three digit NAICS level is a compilation of the more detailed cellular levels (grouped by industry code and region) where the birth/death model figures are applied. This suggests that the birth/death model derived estimates are added to the total jobs before the seasonal adjustment is made.
Further, the BLS paper discussing the model specifically enjoins us not to consider the numbers generated by the birth/death model as a separate series, saying: "The forecasted monthly amounts from the net birth/death model should not be interpreted as an independent time series or even be characterized as displaying a distinct seasonal behavior." (page 4.)
Finally, this paper suggests that the birth death model numbers and the sample based numbers be treated the same and simultaneously: "The evaluation of the net birth/death forecasts should only be conducted in conjunction with the final estimates, as the model values are complimentary to the sample information. However, the net birth/death forecast does have an influence on the seasonal behavior of the final series that can result in increases or decreases in seasonal fluctuations, depending on the month and the industry. BLS publishes the net birth/death adjustment monthly at an aggregate level to inform the user about the effect the model has on the final estimates." (page 4.)
So therefore it is for these reasons that I think I've faithfully replicated the impact of the birth/death model on the BLS numbers. I do however think you raise valid points on how this model is handled by BLS and those points raise further objections over the validity of the whole exercise on their part.
Best - RT
.
RT - Here are my comments to your latest points:
RT: I think we're just caught up in semantics here. A job actually counted in a survey is a real datum. One not observed but merely estimated is not a real datum. In this case I call into question the basic premise of the estimation method. Estimation methods should be based on real world correlations brought about by causitive factors. Accidental correlations can vanish at any point in a time series. This model assumes the patterns of business birth and death are similar in both boom and bust of the business cycle. I say this because the model was backtested in boomtimes and worked great, then applied to work in the bust period of this last business cycle. I don't think assuming that business deaths and birth behavior is the same during boom and bust is a legitimate assumption to make. Further, with increasing globalization, the nature of employment and hiring is fundamentally changing as we speak. I don't think this model captures that at all. How can one assume that a business death means a business birth steps in, in the same region in the same industry in an era when a business death could well mean that the enterprises' function in the economy is now performed in India, or Mexico, or by a computer program run out of the company's main office?
FTM: Keep in mind that the model is applied at below the sector level so even in the nineties when the model was tested live on a sector basis there were certainly sub-sectors where establishments were contracting. Also you can be certain that BLS back-tested the model on data extending back before the nineties. It is only their live testing that was limited to the late nineties. Therefore the model was not conceived without the benefit of negative cyclical periods. Remember the second component of the birth/death model corrects on a sub-sector level for any consistent errors in the prediction of births based on deaths. Given this correction there is no reason to think the model is biased based on the use on deaths as a predictor of births. Also if many births are not really births but reorganizations, you can see why deaths are a good predictor of births.
RT: The fact that the birth/death model is suddenly detecting four times the imputed jobs than were detected by the same model (on paper) in 2002 or 2001 suggests that something has gone seriously awry. Consider: in "Accounting For Business Births And Deaths In CES: Bias Vs. Net Birth/Death Modeling", BLS reports that in March, 2001 through March 2002, 154,000 net jobs were imputed by the model. For the following year, 156,000 jobs were imputed. Suddenly, when the Birth/Death Model went live in March 2003, the subsequent year saw jobs imputed by this method jump to 642,000, a fourfold increase. In the two months since March 2004, we've seen an additional 435,000 jobs imputed. A rate that, if extended to the coming year, will be another fourfold increase! )
FTM: The numbers you cite are for the total BD jobs for only the combined mining, manufacturing and construction sectors. These sectors in 1994 (the only figures conveniently at hand) comprised only approximately 25% of US employment. So the current numbers seem entirely consistent with those you cite above.
RT: Not at all. Your 40% number comes from the time period of January through May 2004. My 85% number comes from March 2003 through May 2004. Thus they aren't directly comparable. Further, it's a fact that the nonseasonally adjusted numbers from March 2003 through May 2004 are widely different than the seasonally adjusted changes. Using unseasonally adjusted numbers one comes up with over twice as many new jobs as there were reported by the seasonally adjusted series (2.7 million versus 1.3 million) in the 3/03-5/04 period. This is the reason why our numbers are so far apart.
FTM: The 40% number holds for the entire period and either calendar year sub-period. The reason the adjusted numbers differ so much from the unadjusted is purely a function of sample period starting in March and ending in May. It merely shows the extreme seasonality in the numbers.
RT: We got caught up in words here again. The BLS reports their total product of both steps as "Net Birth/Death Adjustment". It is the effect of the total product of their two-step process that we are examining here.
FTM : Yes, I agree I’ve confused things here with terminology as I was relying on an old BLS paper. From now on I’ll refer to the two components of the “Net Birth/Death adjustment” as the imputed component and a residual component. My original point is still valid. Either of these two components can be positive or negative. And we don’t know when one is or isn’t because we only know the entire “Net Birth/Death adjustment”.
RT: This is dangerously close to a tautology. Few are claiming slowing employment growth merely because very few are like you or I torturing these numbers to death to find truth and are instead taking the huge numbers out of BLS at face value. Frankly, if the BLS are "imputing" over 700,000 new jobs by virtue of what they see in business deaths, what that suggests by definition is that actual sample surveys revealed to them that about 700,000 jobs were eliminated due to business deaths in the last four months.
FTM: If you remove the BD jobs from the unadjusted numbers and employment still growing steadily this spring. This is an environment where the model “may” produce downwardly biased estimates, not upwardly biased ones. Also at the risk of sounding like a broken record, the estimate is not just a function of deaths, there is also the residual correction component. So if a sub-sector (say software consulting or some other sector harmed by off-shoring) had a consistent history of more deaths than births, the model will account for this.
The actual numbers don’t show 700,000 jobs lost to deaths. The jobs for births are imputed from deaths based on the employment growth or shrinkage in a sub-sector. This is an important point. Deaths only give the number of establishments going under not the impact of those establishments on employment. So if employment is declining by an average of 1 job per establishment in a sub sector and 10 establishments go under, that results in an imputed estimate of -10 jobs. The imputed number of jobs based on business deaths can be negative independent of the residual correction. This shows that economy wide it will be very difficult for the birth/death number to be positive unless there is employment growth in existing businesses.
RT: Perhaps it is logically flawed to apply the same seasonal adjustment to "both series". However, I find no evidence on the BLS site that they did anything different and much evidence that indicates that this is exactly what they did. First, looking at the BLS's explanation of seasonal adjustment of the employment statistics, we find no discussion of difference of treatment of birth/death model generated numbers at all. Just a general discussion of taking totals found at the three digit NAICS level and applying the seasonal adjustment program on them. The three digit NAICS level is a compilation of the more detailed cellular levels (grouped by industry code and region) where the birth/death model figures are applied. This suggests that the birth/death model derived estimates are added to the total jobs before the seasonal adjustment is made.
FTM: You are correct that BLS just lumps the birth/death numbers in with those from existing businesses prior to any seasonal adjustment. But there is nothing wrong with this approach.
Your attempt to back out a seasonal adjustment for the BD number is where the problem arises. The BLS seasonal adjustments are built upon the composite series. So when you try to apply those seasonal adjustments to a series without the BD numbers you are in essence assuming the BD numbers have contributed no seasonal variation to the composite series. Hence you get results which are the same as if you make no seasonal adjustments to the BD numbers.
RT: Further, the BLS paper discussing the model specifically enjoins us not to consider the numbers generated by the birth/death model as a separate series, saying: "The forecasted monthly amounts from the net birth/death model should not be interpreted as an independent time series or even be characterized as displaying a distinct seasonal behavior." (page 4.)
Finally, this paper suggests that the birth death model numbers and the sample based numbers be treated the same and simultaneously: "The evaluation of the net birth/death forecasts should only be conducted in conjunction with the final estimates, as the model values are complimentary to the sample information. However, the net birth/death forecast does have an influence on the seasonal behavior of the final series that can result in increases or decreases in seasonal fluctuations, depending on the month and the industry. BLS publishes the net birth/death adjustment monthly at an aggregate level to inform the user about the effect the model has on the final estimates." (page 4.)
FTM: This is all true. We really shouldn’t try to independently seasonally adjust the BD numbers. They are not independent of the total employment series. If you want to estimate the impact of the BD numbers, it makes the most sense to just say they contribute 40% of the seasonally unadjusted new non-farm employment and leave it at that. Also the link you provided above (which fyi isn’t the paper cited) states on page 16 that over the sample period (1990:1994?) establishment births and deaths comprised 37% of the annual change in jobs. So this is one more bit of evidence that the current BD contribution is not disproportionate.
All of us would be a lot better off if BLS did a better job of explaining their methodology and didn’t publish any number unless it can also publish a seasonally adjusted version.
Cheers
FTM
FTM: >>All of us would be a lot better off if BLS did a better job of explaining their methodology and didn’t publish any number unless it can also publish a seasonally adjusted version. <<
I think our discussion is proof of this. It will be interesting to see June's number's this Friday. -RT
This discussion is continued in the comments to my latest post "June Jobs Report: A Reality Check". -RT
Funny how all those "imaginary" jobs have a measurable effect on the rest of the economy. Or do they? Just how much hard data can a conspiracy theory withstand before it finally implodes? http://thedeadhand.com/blogs/jscroft/archive/2004/08/31/474.aspx
(Cross posted by me on http://thedeadhand.com/blogs/jscroft/archive/2004/08/31/474.aspx )
Interesting data. You left out one possibility though. Possibility #4 is that correlation is not causation, and even if there was causation, you may not know what it is. In this example you correlate the results of your firm's sales with general employment trends, thinking there is a direct causation. There are so many artifacts and problems with this that meaningful analysis is impossible. Here's just a few: (1) You presuppose that the newly unemployed can afford to buy any kind of health insurance. Most cannot. The numbers of uninsured have risen by millions over the last few years. (2) You presuppose that the re-employed are offered health benefits. Recent trends show that jobs being created to replace ones lost are less likely to offer such benefits. (3) High deductible plans sold in conjunction with HSAs are getting much more popular. Your site does not appear to offer such and your sales may be suffering on that account. There are lots of other problems, but I think you get the point.
Since you felt the need to lecture about scientific methods, I'll send it back to you: when you want to measure something, it's always best to attempt to measure that something (jobs) directly rather than to measure some other thing altogether (sales success of your own company). And, even when succeeding in apparently measuring what you want to measure, you often have to design and redesign your study repeatedly to factor out all the variables that get in the way of what you think you're examining. One of my favorite lectures on the subject is by Richard Feynman, called "Cargo Cult Science" accessible here: http://wwwcdf.pd.infn.it/~loreti/science.html Certainly I would be as rigorous as possible with one's own examinations before one elects to be as derisive as in your post above.
In regards to your point about observing all phenomena when forming a theory, your point is well taken. Because of that point, I was surprised to see your examination of one small microeconomic slice (one company's insurance sales) and the neglect of volumes of economic data that show that yes indeed those 1+ million jobs were not reflecting their purchasing might in the economy as seen by the slowing GDP numbers, the sharply reduced July jobs report (and retroactive slashing of earlier estimates), the declining consumer confidence numbers and the like. Virtually all of the economic projections made in the financial press in May and June were based on crunching numbers which included the robust jobs growth reported. Lo and behold those estimates came in wildly optimistic in subsequent experience of consumer spending and other economic activity.
In addition, you resort to the fallacy of the straw man argument for virtually your entire discourse here. For the record, I do not subscribe to a "conspiracy theory" about the BLS. Quite the contrary, here is what I did say on this subject:
"Here's what it doesn't mean: It doesn't mean the BLS are a bunch of lying scoundrels. The birth/death model is a serious attempt to measure something real, albeit undetectable: the number of jobs created by new enterprises that haven't checked in with the unemployment insurance offices that the BLS samples to collect it's establishment job data. The BLS has been completely straightforward in it's admissions of the shortcomings of the statistical estimation techniques it uses to guess at the undetectable. The BLS has also said that birth/death modeling stinks at detecting changing trends. (I'm liberally paraphrasing.) It also doesn't necessarily mean that all the jobs imagined by the birth/death model don't exist. SOME of them do, since there has to be some new businesses out there that DID create some jobs that haven't reported in to their state bureaucracies yet. "
Finally, I want to thank you for the courtesy of pointing me to your blog post by posting your url on my comment page. That allowed me the opportunity to see what you said and to respond. That's a courtesy that's virtually unknown in the blogosphere and I appreciate it. Who knows? I might even get a health insurance quote from you sometime.
Kind Regards,
Richard Torgerson
You're welcome... but don't expect disclaimers to get you off the hook. Your straw man is on fire at http://thedeadhand.com/blogs/jscroft/archive/2004/09/01/480.aspx.
Aww, Richard, come on... don't you want to see how the arithmetic works out? This is the rational argument you've been waiting for!
Jason G. Williscroft
The Dead Hand
I was discussing politics with my boyfriend and he referred to http://www.moveleft.com/moveleft_essay_2004_06_07_fake_job_numbers_from_the_bush_administration.asp and the following article and comments. I know nothing about economics but am I to understand that between two obviously learned men as yourselves that there is the agreement that between 40%-85% of the created jobs reported are imaginary? 40% is downright appalling. At 85% we are truly not moving especially since the numbers are low to begin with. This only confirms my belief that multinational corporate representation in our government is destroying this country. We need to get their numbers out of our system because they represent global not American economy and allows the government to give false information to the American people. The US government is taxing the American people not the entire global world. Why should I support the entire global community if I can't vote in their elections? We give Congress and the Presidency the privilege of being the most powerful and influencial people in the world. I think they've forgotten who they work for and that's why this country's economy is such a mess.
I was discussing politics with my boyfriend and he referred to http://www.moveleft.com/moveleft_essay_2004_06_07_fake_job_numbers_from_the_bush_administration.asp and the following article and comments. I know nothing about economics but am I to understand that between two obviously learned men as yourselves that there is the agreement that between 40%-85% of the created jobs reported are imaginary? 40% is downright appalling. At 85% we are truly not moving especially since the numbers are low to begin with. This only confirms my belief that multinational corporate representation in our government is destroying this country. We need to get their numbers out of our system because they represent global not American economy and allows the government to give false information to the American people. The US government is taxing the American people not the entire global world. Why should I support the entire global community if I can't vote in their elections? We give Congress and the Presidency the privilege of being the most powerful and influencial people in the world. I think they've forgotten who they work for and that's why this country's economy is such a mess.
Post a Comment