<< Back Home
How to Build Your Own Database!
or "That's Right! XML, CSS and XHTML are now freeing the Web World of Proprietary Database Systems!", 6/26/06
It's Your Data, Right?
(This article will be brief, till I have time to give you some code. Till then, I hope what I am describing helps those of you wondering how to create your own database).
DATA....the world is all about data! It has been for some years now! But if you look closely at the history of data and the database you will find not much has changed. Data changes constantly, but the framework behind it has not. That's ok, but in some ways, not so ok. Why? Because we are left with a landscape where a handful of global corporations control the creation of software that serves our data, and the languages that communicate with that software. Trust me, thats ok....in a general sense. It's probably ok also because we are all so familiar with Oracle, and SQL Server, amd MySQL database systems, right? And we know how to communicate with thsoe systems consistently. Consistency is a good thing....very good! But something small down deep in my gut bothers me, and has sense I created my first Access database. And even now as SQL Server 2005 is my database of choice, something still bothers me....Its like a deep down doubt in the back of my mind, that something so wonderful as these databases and the languages they use just might eventually not be the way we all need to use our data and manage it. I cannot quite pinpoint what it is about these relational databases and how we use them, but something for a long time has bothered me. Let's get down to the issue here...
In order to get to the meat of this article and help you build your own database, I want to make a point. The point is this: Why is setting up, using, constructing, architecting and modelling data such a GIANT HEADACHE and undertaking? I mean, I know many of you LOVE your relational schemas....I can understand it. Believe me. And we all have our own rich and complicated schemas we have developed over many years of playing with data and database modeling. But here is my issue....why is it such a huge headache to set up a database? Why are our schemas getting so COMPLEX and CONVOLUTED? Whats the purpose of that? Wasn't it Einstein who said something about genius and simplicity? How is it that he could use such simple visual concepts to tell us about relativity and time travel? What is it when we have a room full of smart people that none of them can really help us understand what they are trying to do, but suddenly there is always that one person that is so bright and almost child-like that has this amazing ability to sum up everything a group of people is working on into a single sentence and explain to us exactly what we need to know about that subject? Why is it so hard for us to use data, build systems that help everyone manipulate data, and explain to the uninitiated how to use a database themselves? Why is using data getting so complex???
I guess that issue I have with todays database landscape is its TOTAL LACK OF SIMPLICITY!!!
Seems to me, that if I want to open up a browser and type some text and send that to a friend, or a family photo, that some simple little system should allow me to not only do that (ie. like email does now) but also SAVE that exchange or data somewhere in my own form of database I create, so I can see it again without using someones proprietary software system and/or database. I guess what Im saying is...why is data management and data use and data inside software so complex, used so poorly, and so disparately from other system, and so wrapped around a foggy arrangement of desktop, network, client-server, and web based solutions none of which allow me to get to that RAW DATA or manipulate it without buying or connecting to some expensive, complex, system?
Let me try and explain again what Im saying, as I know I am not totally clear....
Question: Why is it so hard to use data and databases????
Or.....How can the average user sitting at home begin to manage their own data???
That is an age old problem....how to help people manage and use their own data. The way we have begin to resolve that is via email, and via portals on the web, and via Microsoft Office, and a wealth of simple to complex content and time management software tools. As we move into the business world, we have our gigantic SAP, and Siebal system, and IBM management systems, and our Micrsoft Sharepoints and CRM solution and B izTalk servers. We have more and more complex languages and tools to access those system....like .NET and JAVA, and C++ and C#, etc. etc. And the databases that tie it all together have their own languages and objects that allow us to talk across the various layers to get to that data. Data and the systems that allow us to store and manipulate that data is getting farther and farther away from the ordinary person, and more in the hands of giant tool stes and languages and systems and databases that only seasoned experts can access. My question is.....WHY??????
Why Must data and storing data be such a complex and convoluted system? How did we get to this point? It's your data, right? Why are so many companies making it so hard for you to get to your data???
Bringing Your Data Closer To You
One of the things I absolutely LOVE about the web and being a web developer is the fact it is SO accessible to so many people in so many ways. Beside the information it provides, it also is reasonably accessible as far as being able to create your own web pages! You dont need much to have a website these days and create a web page others can see. Its just a text file, actually with a little extra code in there, if you really think about it. Softare engineers call that a "static web page" but its not static if you are typing text "data" into it every week, is it? And having a web domain for others to get to is cheap and easy to do....if its not a few dollars a month its totally free! To type your first web page you dont have to own any software. You can type HTML in Notepad or some other free text software on your computer and in no time have your own website and content up online. Its NOT data really....a simple web page.....or is it?
Ok, many of you may be laughing as an HTML page is so far from say building a dynamic ASP.NET or JSP data-driven web page, its not even funny! But even though I personally use and enjoy the software tools and proprietary languages I use at work (ie c#, ASP, etc), there is something "beautiful" about typing up your own html page and sticking it on the web! And that process of doing that hits very close to my heart, and is the main point Im getting to in creating your own database....
Its my feeling that is now possible to use the web to manage and create your own data, and database WITHOUT using any software from Microsoft, or Sun, or IBM, or even Google free toolset. Im not saying those languages, system, databases and tools dont have their place. What Im saying is that there are some new languages and open standards available that if used carefully, do allow you to create web pages and systems that would allow you to essentially not only use and view your web pages as content, but also see that content as managed data. And from there, having pages in what many called "unstructured" data sets, begin to build your own website around your very own database! That's what I have begun to do!!! Its about bringing your data closer to you....allowing you more immediate access to it not only via a web browser, but using web standards that allow you to store your data online as both presentation AND in a storage medium. It about bringing data back closer to the source....which is YOU!!!!
A New Database Concept
Data does NOT have to be stored in relational databases!!! That's right! Many people dont see data as data till its in a rigid structured medium somewhere. But lets look at what we are talking about.....data is nothing magical.....its just a bunch of alphanumeric characters folks! Thats it! Whether its encrypted or not, whether its stored as a binary or not....still basically presented to you, the user, as numbers and letters! So I ask, whats the big deal about databases??? Crazy question, right? The deal is that you do not have to use relational expensive complex databases to manage data! WOW...what a revelation! There is another way....maybe not a BETTER way always, but a way you can manage and create and access your own alphanumeric characters right there with your web pages on the server. That way is native
XML storage files!
Most people think of
XML as a language, or a framework, or a technology, or unstructured data. Its all those things and I think, in some ways, none of them. To me, "Extensible Markup Language" is a
way of managing data or rather some text characters you can type into Notepad or other text editor that instantly make your data actually a database! Now, that may go against the formal defintion at the w3c.org, who manages this technology and sets its standards for the world. But really, XML seems to be a way for your average user on a computer to type in some code and have a small piece of data, which believe it or not, other programs can nearly ALL ACCESS INSTANTLY! Is that cool or what? How many of you know of a browser, or software or database or system that if you sent it an XML text file, could somehow read it and get to its data? Allot of system can do that. Thats why people see XML as a form of data "exchange". Really its not....that to me limits its power. I believe the power of XML is its ability to STORE DATA!!! Seems crazy but thats what it does! Does not matter whether its storing your text content in a text file on a computer, or in memory, or being shipped via a SOAP packet between two computers which are exchanging data....its all stored data!
An example of XML for you...
<your_xml_data>Hello World!</your_xml_data>
Now XML has a sister...her name is XHTML. XHTML IS XML, but also HTML. Its both! So, guess what, you can type your old web pages using HTML like you may know how to do, but if you follow the rules of XHTML and XML structure in your Notepad, its also....guess what.....DATA!!!!! Thats right! It is now possible to have a web page that looks pretty (using CSS), is content rich and structurally informative (HTML) and is a
mini-database of sorts (XML). Using XHTML you can now have what I consider structured web data and your web pages could be accessible as data from an XML parser. Now it gets better.....
Its also possible now to not only store XML on a website as plain data, but add a page called XSLT to it and a style sheet (CSS) and send that to a browser. If properly typed using a text editor, all three can be sent to a users browser, and the end result is a pretty web page just as you designed it with your data content!!!. Browsers Internet Explorer version 5-7 and Firebird/Mozilla/Netscape can also take all three files from your server and create a complete web page....no database OR dynamic servr-side pages required! Its that way-cool or what. Suddenly, if you think about it, you do not need to buy anything to have a web page! Its all text files now, thanks to the w3c.org and the implementation of these new
Web Standards! All you need is a website and three text files!!! Thats it! What more do you need! You have your web page, your content, your dataset, and your graphic design via CSS pages and images. Thats it!
Now to the meat of this article about data.....my point about XML and this article is to show you that DATA DOES NOT HAVE TO BE WRAPPED INTO A COMPLEX RELATIONAL DATABASE TABLE! You amy say, but Mitch, thats ok for very simple static web pages, but what about managing complex data sets in your website.
Here is my point....data using XML and its sister XHTML and CSS with XSLT allows you to have everything!!! You have your web design, your content, and your data all together on the server. The data could be stored in either an XML file you transform with an XSLT and CSS or all in one in a simple XHTML file. Your data is now more accessible than its EVER been before. The question now becomes....how can I manage these XML or XHTML files myself. The answer is...store ALL YOU DATA IN XML FILES and build a system to MANAGE THOSE FILES AS A DATABASE! Thats it!
Ok, so how do I do that....the the $64,000 question right? Well, Ive build a little server-side system to do that. It uses one small built in Microsoft technology and ASP web pages to do that. But you could do that using PHP, or JSP or any number of system YOU LIKE. You could also have NO MANAGEMENT SYSTEM at all, but just have a simple folder or XML files, which are sent with XSLT and CSS files as people link to your web pages. Or store them in XHTML with id's on each element, and other systems could use those web pages as xml data. The point of this is to show you that storing XML on the web server in its native form is a very viable and very cool alternative to storing and using data via a complex proprietary database system. The key from here is to use this XML text data in files that you organize as files using a simple file-based system, but server as XHTML using the technologies described above. But the coolest idea is figuring out how to manage these text files as XML? How to use them beyond XSLT and the web server? How to manage them as a single unit? How to manage them as files on the server? How to write to them and read them and parse them? How to store data in them and grow them? Basically, I want you to learn how to use the native XML text file as a part or a file-based database! Doing so, means you can now write your own database on your website! And that technology detail will be the discussion I will add to either here or in a future article. Let me give you a taste or that by listing the things you can do to create this database yourself:
How to Create Your Own Database
- Pick a web server scripting language to use: Either ASP or PHP are good. You will need a server-side language thats lite and simple to manage your xml files and write to them Dont use heavy object oriented langauges like ASP.NET or JAVA as they have way to many requirements and limited to compiling and complex.
- Pick a web server that supports your scripting language and has the XML API's and parsers to support your language: Windows for ASP or Linux/Apache for PHP.
- Pick a simple and freely-available XML Parser thats built into your language of choice. I like to use Microsoft easy-to-use MSXML parser. You can use XALAN or PERL's XML parse for PHP or some other. Key is to avoid compiled langiages, proprietary COM installs and objects and third party licensed vendor tools. USe whats built into all web servers and/or the Operating System where you website is hosted. Installation free is key!
- Using your scripting language and finding a free XML parser built into the computer that hosts your server, all you need to do now is build small script to load in the XML files on your web server and get the data, the calls to change and save data to them. Thats it!
- Remember, your script pages and use of the XML Parser is not a big deal....you are still dealing with text files, essentially! They may not be read that way in a parser, but its essentially whats happening. ALso, dont let people think its flat-files you are using....its not. Its XML! And its structured data to me Its a database you created yourself! How cool is that!
You essentailly, can now go from there and "get fancy"!. Build a web-based portal page to login and add data to your xml files. Build fancy XPATH queries using your Parser object in your scripts to look for text data and write to web pages. If you see where Im going, you may get a tiny glimse into something truly great happening. Suddenly, you are managing your data WITHOUT A DATABASE! Or, actually, YOU HAVE JUST CREATED YOUR OWN DATABASE! Sure, its not fancy or fast or come with all those cool "JOINS" or QUERIES, but hey, you are now free from Oracle, and SQL Server, and MySQL! You are now managing your own data on your own website using simple text files, and some script pages on a web server. It may not sound that cool, but as soon as you realize you have just created a website of your own WITHOUT needing a database or database license or complex technology, you are fully empowered and now capable of envisioning a future where I have been heading myself....using MY DATA MY WAY, free of other's databases and websites. Manage data at your fingertips on your server and see how empowering it can be!
- Mitchell Stokely, USA
