Does Cognitive testing and training work?

[bpesta22]

Claus I did answer your questions-- you either don't accept them or dont understand them.

IQ tests indeed rank people exactly like personality tests do.

The Big 5 personality test forces a bell curve just like IQ tests do. It takes 5 factors though to measure personality; just one to measure g.

***

RT distributions are not normal because there's a ceiling effect on speed (can't do anything in zero time) but no floor (something that should take 1/2 a second may occasionally take 2 seconds).

Suppose I concede that IQ is anything but normally distributed. Would that change any of the major findings? Ranking people by IQ scores correlates:

.50 with grades
.55 with years eduction
.33 with income levels
.50 with job performance
-.20 with teenage pregnancy rates
-.20 with juvenile delinquency rates
.50 with how accurate you can judge which of two rapidly presented lines was longer
.50 with the SAT, or GRE, or GMAT, or pick any assessment exam.
.70 with trainability

the list goes on and on, so much so that g becomes "the most powerful variable in social science".

These correlations exist whatever the true distribution of IQ.

 

[CFLarsen]

Claus I did answer your questions-- you either don't accept them or dont understand them.

I'll hum them for you:

• Are you saying that there is no correlation between young children with a high IQ and those who later end up with long educations?
  
  
• Can you explain why you don't get an overall higher IQ score?
  
  
• Isn't the explanation to the question "IQ is assumed to be a normal distribution" is "It forces the test to produce a Bell curve" circular reasoning?
  
  
• Would you be convinced by the MMR-example above?
  
  
• At what time in history did IQ data become solid and sound?
  
  
• Do you think the conclusions in The Bell Curve are correct?
  
  
• Are you saying that blacks are less intelligent than whites because they were born that way?
  
  
• Since you are giving conflicting answers to the question "When should we teach rocket science", could you give a final answer?
  
  
• Is it a waste of resources to let an octogenarian get an education from a university?

IQ tests indeed rank people exactly like personality tests do.

The Big 5 personality test forces a bell curve just like IQ tests do. It takes 5 factors though to measure personality; just one to measure g.

If personality tests rank people, which is ranked higher: Being introvert or extrovert?

RT distributions are not normal because there's a ceiling effect on speed (can't do anything in zero time) but no floor (something that should take 1/2 a second may occasionally take 2 seconds).

Suppose I concede that IQ is anything but normally distributed. Would that change any of the major findings? Ranking people by IQ scores correlates:

.50 with grades
.55 with years eduction
.33 with income levels
.50 with job performance
-.20 with teenage pregnancy rates
-.20 with juvenile delinquency rates
.50 with how accurate you can judge which of two rapidly presented lines was longer
.50 with the SAT, or GRE, or GMAT, or pick any assessment exam.
.70 with trainability

the list goes on and on, so much so that g becomes "the most powerful variable in social science".

These correlations exist whatever the true distribution of IQ.

Don't you see the circular argument here? A lot of these have to do with education - and the education you get largely determines your income.

It's like a self-feeding system: The input is largely determined by IQ, so why is it a surprise that the output also is?

 

[bpesta22]

* Are you saying that there is no correlation between young children with a high IQ and those who later end up with long educations?

No; I'd need to look it up, but I suspect the correlation between at least middle childhood and adult IQ is around .70. I'd guess the best predictor of your IQ is your IQ earlier in life; with the second best predictor being your bio parents' IQ.



* Can you explain why you don't get an overall higher IQ score?

In my sample, I do-- the mean was around 24 where 22 is average (and the SD is 7). Not a huge difference in IQ between our college students and the general population. Given that we're a large urban public institution; I don't find that too surprising.

* Isn't the explanation to the question "IQ is assumed to be a normal distribution" is "It forces the test to produce a Bell curve" circular reasoning?

No, because that addresses only the practical reason for making an IQ test to follow a bell curve. The theoretical reason hasn't been posted. It's complicated and depends on IRT. I mentioned that before, and I'm not sure it would be worth my time to post the answer here (especially since I fell that much more straightforward explanations on my part have done nothing to help on your part).

* Would you be convinced by the MMR-example above?

Honestly, I didn't really understand it.

* At what time in history did IQ data become solid and sound?

Answered above. Go back through the history of any young science and you will find dead ends; perhaps even embarrassing ones from the stand point of today's knowledge. But those are indeed dead ends in the sense that no one doing current stuff in the area is even talking about it. Gould picked exactly those dead ends.
It's tantamount to criticizing modern bio because 100 years ago (or whenever) Lamarck's ideas were popular.

We can write a devastating treatise today on the problem with Freud's theory. So what. No one's researching it or talking about it anymore (ignore the fringe / pop psychologists and check out research journals related to counseling / clinical psych). Freud's dead. So too are the strawmen Gould attacked. To kill a corpse is not a scientific advance.

* Do you think the conclusions in The Bell Curve are correct?

I think the data the report have been replicated exhaustively both before and after the book was published.

The conclusions: I disagree with the social policy stuff. I can think of no thing in the environment that might cause the difference, with the exception of pre-natal environments or perhaps very early post-natal.

I think race has a biological component (even if we can't define what that is yet); and I think IQ differences are biologically based. Whether the biology was affected by genes or environment, I dunno, but I think the evidence for the environment is very weak, despite nearly 100 years of trying to find it.

* Are you saying that blacks are less intelligent than whites because they were born that way?

This is nonsense. There's a group mean difference of about 1 sd favoring whites. The overlap in distributions shows millions of blacks smarter than 10s of millions of whites in the USA. I've made this point many times before.

Not sure about "born that way" but I do think biology has something to do with it (see previous answer).


* Since you are giving conflicting answers to the question "When should we teach rocket science", could you give a final answer?

Sure, but I need more parameters. What's it cost to teach rocket science? Are there any opportunity costs (would the pupil be better served spending his time learning something else?)? Would we have to give seats up for those more likely to learn it ? Is the person learning it because he/she wants to be a rocket science or just because of intellectual curiosity?

* Is it a waste of resources to let an octogenarian get an education from a university?

Nope. We have project 60 here where 60+ people can audit courses for free. No where have I ever implied that old people can't learn (my ph.d. is in applied cognitive aging psychology). That said, it's empirical fact that the fluid type measures of g decline with age.

 

[Fiona]

The theoretical reason hasn't been posted. It's complicated and depends on IRT. I mentioned that before, and I'm not sure it would be worth my time to post the answer here (especially since I fell that much more straightforward explanations on my part have done nothing to help on your part).

Your "much more straightforward explanations" are not explanations, so far as I can see: they are tautology. The theoretical reason why you assume normal distribution for this is really important. I am sorry you don't have time to post it, since nothing you have said seems to carry any weight without it.

 

[CFLarsen]

* Are you saying that there is no correlation between young children with a high IQ and those who later end up with long educations?

No; I'd need to look it up, but I suspect the correlation between at least middle childhood and adult IQ is around .70. I'd guess the best predictor of your IQ is your IQ earlier in life; with the second best predictor being your bio parents' IQ.

Look it up and get back when you know.

* Can you explain why you don't get an overall higher IQ score?

In my sample, I do-- the mean was around 24 where 22 is average (and the SD is 7). Not a huge difference in IQ between our college students and the general population. Given that we're a large urban public institution; I don't find that too surprising.

Yeah, but you are not an institution that covers the broad population.

Unless, of course, it doesn't require a higher intelligence than average to enter your institution. But then, that would totally undermine the idea of segregating people based on their IQs when it comes to education - wouldn't it?

* Isn't the explanation to the question "IQ is assumed to be a normal distribution" is "It forces the test to produce a Bell curve" circular reasoning?

No, because that addresses only the practical reason for making an IQ test to follow a bell curve. The theoretical reason hasn't been posted. It's complicated and depends on IRT. I mentioned that before, and I'm not sure it would be worth my time to post the answer here (especially since I fell that much more straightforward explanations on my part have done nothing to help on your part).

It doesn't matter what the theoretical reason is, if, in practice, you simply make the IQs follow a bell curve. Then, the theory is to make it follow the curve.

* Would you be convinced by the MMR-example above?

Honestly, I didn't really understand it.

All I did was to exchange IQ with MMR. If you understand your own example, you shouldn't have any trouble understanding this. Try again.

* At what time in history did IQ data become solid and sound?

Answered above. Go back through the history of any young science and you will find dead ends; perhaps even embarrassing ones from the stand point of today's knowledge. But those are indeed dead ends in the sense that no one doing current stuff in the area is even talking about it. Gould picked exactly those dead ends.
It's tantamount to criticizing modern bio because 100 years ago (or whenever) Lamarck's ideas were popular.

We can write a devastating treatise today on the problem with Freud's theory. So what. No one's researching it or talking about it anymore (ignore the fringe / pop psychologists and check out research journals related to counseling / clinical psych). Freud's dead. So too are the strawmen Gould attacked. To kill a corpse is not a scientific advance.

But we can point to why modern biology is a better explanation today than Lamarck's, and when it happened. I don't see any such explanation from you.

* Do you think the conclusions in The Bell Curve are correct?

I think the data the report have been replicated exhaustively both before and after the book was published.

The conclusions: I disagree with the social policy stuff. I can think of no thing in the environment that might cause the difference, with the exception of pre-natal environments or perhaps very early post-natal.

I think race has a biological component (even if we can't define what that is yet); and I think IQ differences are biologically based. Whether the biology was affected by genes or environment, I dunno, but I think the evidence for the environment is very weak, despite nearly 100 years of trying to find it.

* Are you saying that blacks are less intelligent than whites because they were born that way?

This is nonsense. There's a group mean difference of about 1 sd favoring whites. The overlap in distributions shows millions of blacks smarter than 10s of millions of whites in the USA. I've made this point many times before.

Not sure about "born that way" but I do think biology has something to do with it (see previous answer).

But it doesn't matter if a small group of X turns out to have a higher IQ than a larger group of Y, does it? You have to compare equally sized groups.

Still, if you compare the IQs of blacks, and you think that the difference is due to biological reasons, you do find that they are less intelligent than whites. So that is what you are saying.

* Since you are giving conflicting answers to the question "When should we teach rocket science", could you give a final answer?

Sure, but I need more parameters. What's it cost to teach rocket science? Are there any opportunity costs (would the pupil be better served spending his time learning something else?)? Would we have to give seats up for those more likely to learn it ? Is the person learning it because he/she wants to be a rocket science or just because of intellectual curiosity?

No, you don't need more parameters. If you can say that a certain group of people should not be taught rocket science, you don't need more parameters to tell us when we should.

* Is it a waste of resources to let an octogenarian get an education from a university?

Nope. We have project 60 here where 60+ people can audit courses for free. No where have I ever implied that old people can't learn (my ph.d. is in applied cognitive aging psychology). That said, it's empirical fact that the fluid type measures of g decline with age.

You are doing it again: Whenever there is an example of your ideas being wrong, you modify your explanation to fit the example.

g was constant, right up until the moment when it wasn't anymore.

 

[bpesta22]

Your "much more straightforward explanations" are not explanations, so far as I can see: they are tautology. The theoretical reason why you assume normal distribution for this is really important. I am sorry you don't have time to post it, since nothing you have said seems to carry any weight without it.

You guys wont even accept basic empirical facts-- by that I mean things so well established in the literature that no one disputes them.

Examples would be the reliability of IQ tests, which by definition proves that IQ tests rank people and that the ranks are stable over time.

Or, the predicative validity of IQ tests-- the correlations I've cited above and elsewhere showing what IQ scores predict.

If you won't accept those basic facts, why should I bother getting all theoretical?

For example, I don't understand why the bell curve issue is so important to you and Claus. Please elaborate.

Suppose I stipulate that IQ is anything but normally distributed. How would that change anything? On the other hand, how would it be possible to get .90+ reliabilities and validities larger than what you see with any other single measure of anything else. Are people just making these numbers up? Is it a big conspiracy where the evil IQ scientists are faking their data and forcing everyone else to use the tests when we know they're bunk?

It's obvious to me that Claus simply doesn't understand the points I am making. In fact, he's using them to twist my meaning and claim that he's caught me in contradiction. I don't see that discussing it further is going to get us anywhere, but fortunately I've got lots of time and so will likely try to answer the larson list again.

It might go easier if you or claus stipulated what you might accept re evidence, or at least elaborate your world view on what IQ tests are / do (fine if you think they're complete bunk, but is there anything you think scores on an IQ test indicate, measure or predict?)

 

[Neally]

I've found bpesta22 points both well supported and well stated. I'm confused though as to what/where exactly CFLarsen is trying to go with the arguments he is trying to make.

It might go easier if you or claus stipulated what you might accept re evidence, or at least elaborate your world view on what IQ tests are / do (fine if you think they're complete bunk, but is there anything you think scores on an IQ test indicate, measure or predict?)

That would be helpful indeed.

 

[CFLarsen]

You guys wont even accept basic empirical facts-- by that I mean things so well established in the literature that no one disputes them.

Examples would be the reliability of IQ tests, which by definition proves that IQ tests rank people and that the ranks are stable over time.

Of course they are reliable - they are rigged that way!

Or, the predicative validity of IQ tests-- the correlations I've cited above and elsewhere showing what IQ scores predict.

If you won't accept those basic facts, why should I bother getting all theoretical?

For example, I don't understand why the bell curve issue is so important to you and Claus. Please elaborate.

Because of the implications. If we can say that one group of people is dumber than another (blacks vs. whites), we have no reason to do anything about educating the dumber group. We are scientifically justified to be racist.

Fortunately, we are not, because it's bogus from the start.

Suppose I stipulate that IQ is anything but normally distributed. How would that change anything? On the other hand, how would it be possible to get .90+ reliabilities and validities larger than what you see with any other single measure of anything else. Are people just making these numbers up? Is it a big conspiracy where the evil IQ scientists are faking their data and forcing everyone else to use the tests when we know they're bunk?

You are not going to ignore Cyril Burt, are you?

It's obvious to me that Claus simply doesn't understand the points I am making. In fact, he's using them to twist my meaning and claim that he's caught me in contradiction. I don't see that discussing it further is going to get us anywhere, but fortunately I've got lots of time and so will likely try to answer the larson list again.

It might go easier if you or claus stipulated what you might accept re evidence, or at least elaborate your world view on what IQ tests are / do (fine if you think they're complete bunk, but is there anything you think scores on an IQ test indicate, measure or predict?)

No amount of evidence will suffice if the basic premise is wrong, because IQ-tests are rigged the same way horoscopes are.

The point is, what will it take for you to admit IQ tests are complete bunk?

 

[Fiona]

Ok I will try to explain why I am unhappy. As I understand it, IQ is assumed to be normally distributed. Things which we can objectively measure (eg reaction time) are not so distributed, so there needs to be a theoretical basis for this assumption. What is it? Why is reaction time not part of the IQ test? Is it because, once the assumption is made, IQ tests are constructed of items which produce a normal distribution; that a new IQ test is made by testing a number of items; some of them produce the desired result and some do not;and those which do not are discarded? If this is the case, then it is really important to show that there is an independent theoretical rationale for the assumption. Not a statistical necessity, but a reason to believe it is so in some objective sense. It may be that such a rationale exists but that is what I am asking you to explain.

 

[bpesta22]

Ok I will try to explain why I am unhappy. As I understand it, IQ is assumed to be normally distributed. Things which we can objectively measure (eg reaction time) are not so distributed, so there needs to be a theoretical basis for this assumption. What is it? Why is reaction time not part of the IQ test? Is it because, once the assumption is made, IQ tests are constructed of items which produce a normal distribution; that a new IQ test is made by testing a number of items; some of them produce the desired result and some do not;and those which do not are discarded? If this is the case, then it is really important to show that there is an independent theoretical rationale for the assumption. Not a statistical necessity, but a reason to believe it is so in some objective sense. It may be that such a rationale exists but that is what I am asking you to explain.

Fair enuff. I went back and did some reading and I don't think I can give you (or Claus) a satisfactory answer for why IQ is assumed to be bell curved.

The only way to *prove* it would be to measure IQ on a ratio scale, which we don't do.

I think this is the reason why Jensen devoted the last part of his career to a book on using RT (which is a ratio scale) to measure IQ.

The ECTs I use for example are all RT based. Many standardized IQ tests have subtests that are speeded (about half the WAIS is; the Wonderlic is timed at 12 minutes).

I suspect RT is one of the few things we measure in psych that is not bell curved, and as I mentioned before, I think that's for artificial reasons. There's a limit on how fast one can respond, but no practical limit on how slow.

So, rt distributions are positively skewed with many more slow outliers than fast ones.

These are likely just lapses of attention (in most; not all cases). Run a subject on 100 trials of some boring RT task. Perhaps on trial 46, they started day dreaming and didn't notice the display was on until a second had passed. So, this trial takes 1.5 seconds to respond to when all the other ones took .5.

Just a couple of trials like that and RT won't be bell curved.

I don't think it's all that interesting, psychologically though.

Whatever the true distribution of IQ, the way we measure it now accurately ranks people. That is fact, or IQ tests wouldn't be reliable nor could they possible have predicative validity.

But, since the scale isn't ratio (and likely not even interval past the fat part of the curve), the best we can say is you have more of it then me (if you score higher). We can't say that 100 is twice as much IQ as 50; nor can we say that the difference between an 87 and and 88 is the same as the difference between a 105 and 106.

To make these claims we'd have to measure IQ on a ratio or interval scale.

I predict in the next 10 years or so that will happen as ECT batteries will be developed to measure IQ at the level of cognitive processes (versus standardized test scores).

 

[Fiona]

Ok. It is perfectly possible for a test to be reliable but not valid, is it not? Without a rationale for a normal distribution there is absolutely no way to show that an IQ test is valid. The only point of it is to rank people. If I ask you at gunpoint to choose between identical twins and say I will save the one you choose; but if you refuse to choose I will shoot them both: you have no relevant distinction to base your choice on. You may make a distinction artificially, by, say, tossing a coin. But there is no objective distinction you can point to. Can you show me why IQ is not the same as the coin toss?

 

[bpesta22]

Ok. It is perfectly possible for a test to be reliable but not valid, is it not? Without a rationale for a normal distribution there is absolutely no way to show that an IQ test is valid. The only point of it is to rank people. If I ask you at gunpoint to choose between identical twins and say I will save the one you choose; but if you refuse to choose I will shoot them both: you have no relevant distinction to base your choice on. You may make a distinction artificially, by, say, tossing a coin. But there is no objective distinction you can point to. Can you show me why IQ is not the same as the coin toss?

Surely a test can be reliable but not valid, but IQ tests are both.

Lots of different types of validity exist; I guess the two most important would be predicative and construct validity.

IQ has immense predicative validity. It's the most powerful variable in social science. No other single variable predicts as many things as strongly as g does. That data's been around for decades if not a century.

Construct validity has been established too, with Carrol's work on the hierarchy of human abilities (with g on the top), and now with lots of research showing g correlates with biological, physiological, and neurological aspects of human cognition.

The main function of Jensen's g factor is to show that g exists outside factor analysis as evidenced by it's correlations with things like cognitive processes and brain function.

 

[CFLarsen]

IQ has immense predicative validity. It's the most powerful variable in social science. No other single variable predicts as many things as strongly as g does.

IQ is the same as g?

 

[Fiona]

Can you tell me what you mean by "predicative"? You have used it several times so I am assuming it is not a typo for "predictive".

And can you answer my question showing why this is not the same as a coin toss?

 

[dann]

IQ is the same as g?

Well, that is a different question! Sometimes it is, and sometimes it isn't!

Let us repeat the argument so far:
1. IQ tests allegedly measure people's intelligence,
2. but people who know - not the specific exercises, but - the principles that they are based on get better at doing them and therefore achieve higher IQ scores.
3. According to Helmuth Nyborg, however, there are "no known ways of improving intelligence",
4. but the IQ scores refuse to obey this law of nature and continue to rise,
5. and therefore IQ tests do not measure people's intelligence after all, but are blurred, empty, of no value etc.,
6. which, of course, they cannot be since the advocates of IQ would then have to give up their stupid tests and their entire discipline, which, however, they cling to, and therefore ... (go back to 1) and start again) http://www.skepticreport.com/pseudoscience/iq2.htm

 

[Neally]

Can you tell me what you mean by "predicative"? You have used it several times so I am assuming it is not a typo for "predictive".

Looks like a consistent typo:

IQ has immense predicative validity. It's the most powerful variable in social science. No other single variable predicts as many things as strongly as g does.

 

[bpesta22]

Looks like a consistent typo:

More later but I think the usage of the terms vary by discipline. I like predicative better than predictive and so use it.

either means the same thing though-- the correlation between the test and whatever criterion it's supposed to measure.

 

[Fiona]

Thanks for clarifying that part Bpesta22. The words do not mean the same thing, though.

From OED:
Predicative: Having the quality of predicating. affirming or asserting
Predictive: Having the quality of predicting.

Ironic distinction in the context of this discussion, don't you think?

 

[bpesta22]

Thanks for clarifying that part Bpesta22. The words do not mean the same thing, though.

From OED:
Predicative: Having the quality of predicating. affirming or asserting
Predictive: Having the quality of predicting.

Ironic distinction in the context of this discussion, don't you think?

Very interesting! I've been using predicative all my life. About 2 weeks ago, someone called me on it (in a context not related to anything re IQ). I spent a small amount of time googling it. Seems like predictive is more popular than predicative, but enough people use predicative for me to think I'm not making a typo.

It makes me wonder now though if I and the others who use it are wrong-- it should be predictive?

If anyone has info on the origins of predicative in the context of testing, please share.

 

[bpesta22]

Fiona:

The reason why I don't find the distribution of IQ question particularly interesting (to me-- just my opinion) is that the tests obviously "work". By that I mean even though they only rank people, the ranks correlate significantly with many important outcome variables. And, again, g often emerges as the single best predictor of the outcome.

I stumbled across a conference abstract by R Sternberg (very big name in the field, very anti-g). He wasn't questioning the validity of g, rather he was calling for people to stop doing research on it because it's so well established that g is reliable and has "predictive" validity that people are wasting time further studying it. He's getting sick of people replicating and re-replicating the same stuff, and he argues that it's so well established that further replications are a waste of resources.

This to me is a compelling appeal to authority. Read anything by Sternberg and it's clear he's as anti-g as Gould. Yet he admits the points I've been trying to make here (re g being measurable, ranking people accurately, and correlating with many outcome variables).

That really ought to stop the debate at this level. That IQ/g can be measured, and predicts things is an empirical fact.

Debates about what g is and whether it's largely genetic and the explanation for group differences on it are ok. Really, though, the other stuff is figured out and non-controversial among even experts who despise the concept (true experts; not hacks like Gould).