LAK-11 Further forays into big data and critiques of learning analytics

There are a few resources I found while looking at big-data articles that I would like to include in a post, if nothing else as a handy reference for my own memory.   They also seem to be tied in some way to concerns or questions that we should have about learning analytics.  I would also like to address some of George’s concerns in the LAK-11 forum: Critiques of Learning Analytics?

First some resources:

• A web-page about Data Mining and Discovering Knowledge in your Data is a straight forward and fairly simple explanation of data mining – the basic process and concepts. 
• An article by Tim Nigris entitled: Data Danger Lurking in Public Cloud Contracts, in the Cloud Computing Journal points out concerns that should be had by “organizations considering  or already using public cloud services especially for storing data and documents.”  This should be of concern to the general public as well because ultimately – guess whose data and documents they are storing?
• An article entitled Is the google-ification of education underway? is related to learning analytics (and personalized learning in particular).  But even more interesting (and found in a comment of that article) is this article by Maria Anderson : The World is My School.  She speaks to several pertinent “big-data” and internet issues, but with an eye for their potential to transform education.  It’s personalized learning – but probably not like you’ve ever considered it before.  Her idea is quite intriguing. One of her ideas has been around for quite awhile – adding a game layer.  I’m not sure that’s essential to what she’s proposing, but perhaps for some it would make it more motivational.  This is “inquiry” learning taken several steps beyond what I have seen proposed in the past.  In her vision there is no real separation between “school learning” and “life-long learning”.  That artificial separation between school learning and life-long learning has a huge drag on both.  But maybe that’s what Dewey was trying to tell us all along – except this isn’t so much learning by doing as learning by actively inquiring, making inquiry tools and their processes and thinking of paramount importance. 

Secondly a few of my responses to the first six of George’s concerns about learning analytics.  I'm not sure why I feel the need to defend analytics as I also have concerns. Perhaps its really a reaction to "number-bashing" that I've encountered before and usually react to.  Perhaps it's trying for some sort of balance between recognizing weakness and utilizing strengths (and I freely admit upfront that some of my responses may tend to be more philosophical than practical) :

George says: I see several concerns arising in relation to analytics:

1. "It reduces complexity down to numbers, thereby changing what we're trying to understand"

I disagree that reducing complexity down to numbers necessarily changes what we’re trying to understand anymore than reducing feelings or thoughts down to words changes what we’re trying to understand.  Words are abstractions as are numbers.  But somehow we feel that words can express those feelings and thoughts so much better than numbers.  It’s not the nature of numbers or words that makes one any more or less able to communicate what we’re trying to understand – it’s our ability as humans to map concepts to abstract representations (and perceive that mapping) that may be more proficient in using one representation over another.  As an example consider a high-resolution photograph.  As humans we perceive it quite readily, almost instantly.  Yet to be displayed on a computer screen it most definitely is “reduced to numbers” – each pixel being reduced to one of millions of colors each represented by…a number!   Numbers are not necessarily the lesser medium, just harder  in some instances for human perception, yet numbers have advantages in human recognition of equating and ordering much more so than say, words.

2."It sets the stage for the measurement becoming the target (standardized testing is a great example)"

Only if we let it!  The same could be said of anything that we measure.  What is the alternative to measuring?  Not measuring?  Once something is measured, is it somehow more dangerous?  Is it the measuring process that makes it that way, or the act of measuring, or is it the meaning or importance that we associate with that measurement?  One thing we need to get better at is making and interpreting relative measurements (improvements or the opposite) as applied to a human’s endeavor rather than defining standards of absolute measurements which must be applied to all humans in a defined segment.

3. "The uniqueness of being human (qualia, art, emotions) will be ignored as the focus turns to numbers. As Gombrich states in ‘The Story of Art: The trouble about beauty is that tastes and standards of what is beautiful vary so much’. Even here, we can't get away from this notion of weighting/valuing/defining/setting standards."

Again, I disagree that the uniqueness of being human must unavoidably be ignored “as the focus turns to numbers”, partially because of what I explained in my response to #1 – but also numbers (and computational power) can even be tools that can help us detect something “human” from something computer-generated, or even between an idea that occurred to two individuals at approximately the same time, or an idea that has been simply plagiarized.

4. "We'll misjudge the balance between what computers do best...and what people do best (I've been harping for several years about this distinction as well as for understanding sensemaking through social and technological means)."  This is a valid concern, and yet the balance is dynamic – so what a human may do best today, a computer may do best tomorrow.  In the final philosophical analysis, what “computers do” is subsumed by what “humans do” anyway simply because humans created, programmed, and applied computers for their human problem solving.  But what I think you are saying is that too often we have left to computers (and computation) that which could still significantly benefit from “human judgment”, but is bereft of it, whether because of lack of resources, or humanly-deemed lack of importance.

5. "Analytics can be gamed. And they will be."  And there are at least a million other ways of introducing error in the analytics as well. Whether the accuracy of analytics is imperfect due to gaming or any other error, we should never pre-suppose in our analysis that the “numbers” are without error.

6. "Analytics favour concreteness over accepting ambiguity. Some questions don’t have answers yet."

Any analyst worthy of hire would readily admit – in fact emphatically state- that some questions don’t have answers yet, and interpreting analytics always assumes accepting some ambiguity.  If anyone claims to have all the answers (or even imply it) their credibility is greatly at risk!

In conclusion, these concerns are not to be ignored, but on the other hand they should not be used as an excuse to ignore the analytics either.  In some ways they seem to me like product precaution statements – obvious to most, but existing in print for the express safety of the consumers and liability protection of the producers.