Pl read

1. Why o Why?

If you are visiting these writings for the first time, or have not read the entry "Why o Why",

may I suggest you read that first and then read the rest in numerical order?

Wednesday, 12 April 2017

12. The (in)significance of Data and Dates

As time goes by, for the mere mortals amongst us, the dates and times of occurrence of many of (even) the significant events in our lives become difficult to remember. So when one is asked to narrate events from the past, quite often myriad of  dates tend to get associated with those events.
 A meticulously maintained diary will help, but gaps are bound to be there since some dates may not be covered in the diary entries in enough detail.

Reflecting on such cases now, I feel  that independent of the accuracy of the dates, if the event in question seems plausible, one should examine the intent of the speaker/writer in narrating the event in question, and if that is compelling, move on. I believe it serves little purpose to dwell on the veracity of the minutiae in the narration, especially if  there is some reasonable takeaway from it. Throwing out all that is said is like throwing the baby out, with the bathwater.

This is the principle that will guide my writing in these blogs.

Interestingly, in my own research work, this problem -- relating to gaps in the data -- raises its head. 

In my lab's work on smart energy management,  we analyze data from the past to predict and plan for the future, for example, to decide how much energy can be expected to be produced by a particular  rooftop solar panel and whether it will be sufficient to satisfy the  needs of a building. Vagaries of the weather apart, there are issues related to the prediction of the energy needs, the temperature, cloud  cover, etc. 

The problem is that we need to sense and process a lot of information, store them in a database and later analyze them, but some of the information might be lost in the interim.

Some of the non-IT readers among you may wonder: how can a computer forget or  lose data/information. It can, because there's many a slip between when a phenomenon is sensed and when it gets stored by the computer. It might be lost in the wire  between the sensor and the system, it might be wrongly interpreted, some data might get to the computer while others may be lost, the computer may run out of memory, or might get corrupted, etc., etc. You can see that computers are prone to the same disorders that humans are subject to!

Humans can on many occasions fill the gaps by applying their prior knowledge and common sense. We would like to design computer systems to do the same. That is one of the research problems we are working on.

The bottomline is: 

Even data that is incomplete can be useful, if we fill the gaps carefully;  we will attempt to do precisely that.