Too Much Data Just Taking Up Space – To DB or not DB

Thoughts on information, how do you store it? If you download it and keep it in various forms that you cannot reuse later isn’t it a waste of time? Especially if you should know better.

Since entering the Internet Marketing world I did make an effort to keep the data I collected in an orderly state. The problem always was that there seemed to be so much of it and there appeared to be very little similarity between the things that were gathered together. When, of course, similarity was seen I did begin to start using the tools I had at my disposal. I found everything I collected had been a mix of browser bookmarks for links, regular folders for files and Microsoft Excel for data. The really nice thing about Excel is that it allows you to manipulate and update your data side by side really quickly and I never found the need to create databases.

My IT career however was filled with the need for creating databases and all the different means of manipulating data so that I became an expert in Microsoft SQL Server and a few other types.

It makes me wonder now why didn’t I need to go back into that kind of data storage. I guess the ease at which things were changed and manipulated made me cling to spreadsheets but didn’t I ever really need a database behind it at all?

I am now beginning to realise that a lot of my time playing around with the spreadsheet was a kind of laziness. It meant that I didn’t have to think about the actual structure of what I was doing and the relationships between the data I was storing, except for possibly its type, so that I could at least sort the sheet.

Creating a database usually means thinking about structure before you have the data but this later experience has been more to do with collecting it and just finding a place to store it. Useful if it is something that you are going to throw away again later, but even that begs the question of why is it being done at all then? If I gathered data in any place that I never used then that by itself is a total waste of time. On the other hand a lot of the data I have gathered has been of the same type for most of the time, if it wasn’t collected in Excel it would be in any other of the number of applications that I have used. The real issue that is clear to me now is that I have been collecting data in a way that has limited reuse.

Having the knowledge of how to store and manipulate data must be used to prevent wasting time either restoring it or refreshing it. Having data in a structured form, even if you go as far as data warehousing, gives you a powerful view of the trends and relationships within it.

In short, do not waste time collecting data if you are not able to use it effectively, or it only serves to reduce the amount of free space you have on your hard disk.

This entry was posted in Programming. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *