A Data Storm’s A-Brewin’

There is something of a perfect storm brewing in the world of data today. The world is becoming more automated, more connected, more wireless, and more complex. And that impacts the job of the database administrator (DBA). The next wave of database administration requires intelligent automation. I sometimes refer to this as implementing “software scrubbing bubbles” that “work hard, so you don’t have to.” (You remember that commercial, don’t you?)

As more of the tasks required of DBAs become more automated, the DBA will be freed to expand into other areas. One front on this storm is the autonomic computing initiatives that automate DBA tasks. When system software becomes autonomic some of the tasks that previously required manual intervention and procedures are performed auto-magically.

Some aspects of database administration can be simplified by offloading manual, error-prone tasks to the computer. But the “intelligent” aspect combines monitoring, analysis, and correction. Monitoring is easy for the computer to do… as is correction, if you tell it what to correct. The analysis is the difficult part, the “intelligence,” if you will. But intelligent database administration software exists today that (somewhat) embodies the knowledge and analytical capabilities of a good DBA. Intelligent software like this can be used to implement virtual RoboDBAs for specific tasks.

Of course, not every DBA task can be intelligently automated, but increasingly more and more can be.

At the same time, IT professionals are being asked to know more about the business instead of just knowing the technology. So DBAs need to understand the business purpose and definition of the data they manage, as well as the technological underpinnings of the DBMS. The driving force here is predominantly regulatory compliance. This second front of this oncoming data storm will cause DBAs to work more closely with metadata to drive processes for database archiving, data auditing, data masking, data protection, and security to ensure their organization complies with regulations like Sarbanes-Oxley, HIPAA, PCI-DSS, and others.

Regarding the wireless aspect of things, pervasive devices (for example, smartphones and tablets) are pervasive. As they become more adept at business computing, increasingly these devices will interact with database systems. DBAs will need to get involved there to ensure successful data synchronization. And database systems will need to work seamlessly with disconnected data.

Yet another big database trend is technology “suck.” By that I mean the DBMS it incorporates (or sucks up) technologies and functions that previously required you to purchase separate software. Remember when the DBMS had no ETL or OLAP functionality? Those days are gone. Most DBMS offerings provide some level of both. This will continue as the DBMS adds capabilities to tackle more and more IT tasks.

Another trend impacting DBAs will be a change in some of their roles as more and more of the recent DBMS features actually start being used in more production systems. The net result of this perfect storm of changes is that data professionals are absolutely being required to do more… sometimes with less (less time, less money, less staff, etc.)

If you know the technology but are then required to know the business, this is doing more – much more. But the technology, in many cases, is also expanding. For example, DB2 incorporates native XML. Most DBAs are not XML savvy, but increasingly they are being forced to learn more about XML as the DBMS technology expands. And this is just one example.

Additionally, data is growing at an ever-increasing rate. Every year the amount of data under management increases. According to a presentation I attended at IDUG (May 2012) the speaker quoted IDC that data more than doubles every two years.  And in many cases the number of DBAs to manage that growing data is not increasing. Indeed, in many cases, the number is decreasing.

Budgetary limitations can cause DBAs to have to do more work, to more data, with less resources. When a company reduces budget but demands more work, automation is an absolute necessity. Turning work over to the computer can help (although it is unlikely to solve every administrative issue).

Sometimes IT professionals fight against the very thing they excel in – that is, automating work. If you think about it, every computer program ever written was developed to automate someone’s work: the writer (word processing), the accountant (financials, payroll, spreadsheets), and so on. This automation did not put the executives whose work was automated out of a job; instead it made them more efficient. Yet, for some reason, there is a notion in the IT industry that automating IT tasks will eliminate jobs. You cannot automate a DBA out of existence – but you can make that DBA’s job more effective and efficient with DBA tools and autonomic computing.

And the sad truth of the matter is that there is still a LOT more than can, and should, be done in most companies. We can start with better automation of DBA tasks, but we shouldn’t stop there!

Technologies to help companies comply with governmental regulations are being implemented by many organizations. Software to mask sensitive data from prying eyes, to audit data enabling users to determine who did what to which piece of data, and to better protect data are all hot data technologies right now. But more needs to be done. Database security needs to improve and technologies for securing and auditing data need to be more pervasively implemented.

Metadata is increasing in importance, too. As data professionals really begin to meld together technology and business, they find that metadata is imperative. But most organizations do not have a metadata repository fully-populated and up-to-date that acts as a lexicon for business data.

And finally, something that isn’t nearly hot enough is data quality and integrity. Tools, processes, and database options that can be used to make data more accurate and reliable are not implemented appropriately with any regularity. So the data stored in our corporate databases is suspect. According to Thomas C. Redman, data quality guru, poor data quality costs the typical company at least ten percent (10%) of revenue. That is a significant cost! Data quality is generally bad in most organizations – and more needs to be done to address that problem.

With all of these thoughts in mind, are you prepared to weather this data storm?

About craig@craigsmullins.com

I'm a data management strategist, researcher, and consultant with over three decades of experience in all facets of database systems development and implementation.
This entry was posted in compliance, data, Data Growth, DBA. Bookmark the permalink.

2 Responses to A Data Storm’s A-Brewin’

  1. seo experts delhi says:

    As a Newbie, I am permanently browsing online for articles that can help me. Thank you

  2. “Hi! I just want to give you a huge thumbs up for your excellent info you’ve got right here on this post. I will be returning to your site for more soon.”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.