Last night, Kapil and I had a long discussion on the way research arena is for researchers in Software Engineering (my field) and those in Computer Architecture (Kapil's field). These words are excerpts and afterthoughts.
Computer Architecture is matured beyond measure. The outputs of the research in this field have found awesome success. Computers are good. Ya, there's this thing about unending demands. So, there's always a reason to have a better computer than the one we have. But frankly, this research field has delivered. It essentially consists of some practical kind of research, which does take help of complicated math, but in controlled measures. Its takers are big chip manufacturing firms investing billions on innovation. Each individual consumer is eager for new ideas. They come in large number; and a good number gets quickly consumed. Therefore, there has been very feverish research in the past couple of decades. Many people have crowded in. It looks that all that could be thought out, has been thought out. Not that nothing more is left to think. But whatever there is, has almost become obvious to everyone due to the maturity of the field. Hence, there're many researchers ready to pounce on a problem, the moment it appears. If you are a researcher in Computer Architecture, and you notice a problem, you can be sure that there would ten others all over the world who might have noticed the same problem and would have already started working hard on that. To make things worse, many of them might have greater resources than you, in terms of man power and experience and perhaps even sharpness. So, the bottom line : If you have got an idea, you better be quick to take it to the finish. For if you aren't quick, somebody else will surely be, and just when you are about to see light at the end of the tunnel, you will be hit with a bolt from the blue -- a paper coming out from some unknown competitor of yours.
Software Engineering as a research field is very different. Very unlike the beliefs of a lay person, research problems in this area are many, and working solutions, very few. The state of the practice uses archaic methods which further curdles the already messy problem space. Softwares are built and maintained at such breakneck speed that there's no good way of making online studies. Moreover, the problems in Software Engineering are mostly related to a number of -ilibilities as they call them. Maintenability, portability, testability etc. These are as of now immeasurable quality parameters. In absense of proper metrics, what can researchers in this area hope to improve? The field is not so matured as Computer Architecture. Consequently the research efforts are pretty scattered. One advantage of this is that an idea occuring to you has a significant chance of not having occured to you. Disadvantage: you don't have any benchmarks to test the goodness of an idea. If you sound too concrete, you could be blamed with proposing something trivial. If you are too abstract, you could be blamed with proposing something too wild, impractical. Worse: you may be charged with 'handwaving!' Problems are many in this field which are crying for good solutions. However solutions are nowhere close to really alleviating the pain that Software making as a practice is.
Kapil asked the question: 'What evidences occur in history when there was a dire need of a paradigm shift of thinking, and then an invention came and solved the problem.'
We thought, and within our limited knowledge couldn't come up with any such example. There are plenty of examples where there were good ideas, which were displayed just like that. And then they caught the attention of users, and they flourished. But no example could be recollected when a hitherto non-existing technology appeared in rescue of mankind from a pressing crisis. On thinking hard I feel there do exist solutions which involve clever adaptation of existing technology for providing a solution to a crisis. Additionally, I think some examples from the World War 2 could be found where a pure technological solution came in direct response to a military requirement, and it changed the history of the world. But no such technology which emerged in response to a crisis that's common to all. Well, that's the way things are are. Can't complain!
If software engineering research comes up with some breakthrough research results now, it will be an invention of that type -- one in direct response to a crisis. something that doesn't seem to have happened in visible history of science. It wouldn't perhaps be silly to assume that there's not going to be any such breakthrough after all very soon. Such breakthroughs seem to occurs in two extreme conditions: when there's perfect peace, and when there's war. The current scenario is neither of this. Of course, it doesn't seem to be growing any more peaceful every coming day. Perhaps, we'll soon have a war like condition, and then we will come out with the real solutions.
Bits of Learning
Learning sometimes happens in big jumps, but mostly in little tiny steps. I share my baby steps of learning here, mostly on topics around programming, programming languages, software engineering, and computing in general. But occasionally, even on other disciplines of engineering or even science. I mostly learn through examples and doing. And this place is a logbook of my experiences in learning something. You may find several things interesting here: little cute snippets of (hopefully useful) code, a bit of backing theory, and a lot of gyan on how learning can be so much fun.