Everyone in IT Should Serve A Mainframe Tour of Duty

Today’s post is an update on a post I first wrote 10 years ago on my DB2portal blog. The general idea is that everybody in IT would be well-served by learning about mainframes and their robust management environment.

Mainframe developers are well aware of the security, scalability, and reliability of mainframe computer systems and applications. Unfortunately, though, the bulk of new programmers and IT personnel are not mainframe-literate. This should change. But maybe not for the reasons you are thinking.

Yes, I am a mainframe bigot. I readily admit that. In my humble opinion there is no finer platform for mission critical software development than the good ol’ mainframe. And that is why every new programmer should have to work a tour of duty on mainframe systems and applications as soon as they graduate from college.

You may note that I use the word mainframe, instead of the z Systems or z Server terms that IBM is using these days. Nothing wrong with the z thing, but I think there is nothing wrong with the term mainframe!

Why would I recommend a  mainframe tour of duty for everybody?

Well, due to the robust system management processes and procedures which are in place and working at every mainframe shop in the world. This is simply not the case for Windows, Unix, and other platforms. Of course, I don’t want to overly disparage non-mainframe systems. Indeed, much of the credit for the mainframe’s superior management lies in its long legacy. Decades of experience helped mainframers build up the systems management capabilities of the mainframe.

But by working on mainframe systems, newbies will finally begin to learn the correct IT discipline for managing mission critical software. The freedom that is allowed on non-mainframe systems helps folks to learn – but it is not conducive to the creation of hardened, manageable systems.

No longer is it okay to just insert a CD download something from the web and install new software willy-nilly onto a production machine. Mainframe systems have well-documented and enforced change management procedures that need to be followed before any software is installed into a production environment.

No longer is it okay to just flip the switch and reboot the server. Mainframe systems have safeguards against such practices. Months, sometimes years, can go by without having to power down and re-IPL the mainframe.

And don’t even think about trying to get around security protocols. In mainframe shops there is an entire group of people in the operations department responsible for protecting and securing mainframe systems, applications, and data.

Ever wonder why there are no mainframe viruses? A properly secured operating system and environment make such a scenario extremely unlikely.

Project planning, configuration management, capacity planning, job scheduling and automation, storage management, database administration, operations management, and so on – all are managed and required in every mainframe site I’ve ever been involved with. When no mainframe is involved many of these things are afterthoughts, if they’re even thought of at all. Sure, things are getting better in the distributed world – at least better than they were 10 years ago – but it is still far from perfect!

Growing up in a PC world is a big part of the problem. Although there may be many things to snark about with regard to personal computers, one of the biggest is that they were never designed to be used the way that mainframes are used. Yet we call a sufficiently “pumped-up” PC a server – and then try to treat it like we treat mainframes. Oh, we may turn it on its side and tape a piece of paper on it bearing a phrase like “Do Not Shut Off – This is the Production Server”… but that is a far cry from the glass house that we’ve built to nourish and feed the mainframe environment.

IMG_0503

And it is probably unfair to criticize PCs for not being mainframes because the PC was not designed to be a mainframe… but over the years people have tried to use them for enterprise production workloads… sometimes successfully. Sometimes not.

The bottom line is that today’s applications and systems do not always deliver the stability, availability, security, or performance of mainframe systems. A forced tour of duty supporting or developing applications for a mainframe would do every IT professional a whole world of good!

Posted in enterprise computing, mainframe | 2 Comments

Avoiding Deadly DBA Habits

Every now and then I’ll write a short blog post to promote an article that I think is worth reading. This is one of those posts…

Today’s article worth reading is titled The Seven Deadly Habits of a DBA …and how to cure them by Paul Vallee. This well-thought-out list of habits to avoid should be on every DBA’s reading list. It is well worth the short time it takes to read through it.

The article is not deeply technical, nor should it be. Although the article is written from an Oracle perspective, the guidance is broadly applicable to managing any DBMS environment. In the article, Vallee discusses seven tendencies that need to be overcome and offers cures for overcoming them… ’nuff said.

Click on over and read it while you’re still thinking about it: The Seven Deadly Habits of a DBA …and how to cure them.

Posted in DBA | Tagged | 1 Comment

Inside the Data Reading Room – Spring Break 2015 Edition

Welcome to another installment of Inside the Data Reading Room, a regular feature of this blog where I take a look at some of the recently published database- and data-related books. In today’s post we’ll examine a book with a unique spin on data governance, a book on MDM and Big Data, and an A to Z guide on Business Intelligence.

The first book we’ll examine inside the Data Reading Room is Non-Invasive Data Governance by Robert S. Seiner (2014, Technics Publications, ISBN 978-1-835504-85-6).  This book offers an insightful and practical approach for embracing data governance best practices by evolving existing data governance activities. The premise of the book, as Bob so aptly states early on, is that “(a)ll organizations already govern data. They may do it informally, sometimes inefficiently, often ineffectively, but they already govern data. And they all can do it better.”

The book does a wonderful job of explaining the trademarked phrase that is the of this book, “non-invasive data governance,” as well as to guide the reader along the path of formalizing and improving existing data governance practices. The key, according to Seiner, is not to start from square one, but to build upon the data responsibilities that are already being performed within the organization.

If your organization is planning to embark on a data governance project, is looking for help for your existing data governance practice, or simply desires a useful, non-invasive method of managing and governing your data, look no further than Seiner’s informative Non-Invasive Data Governance.

Next up is Beyond Big Data: Using Social MDM to Drive Deep Customer Insight by Martin Oberhofer, et al (2015, IBM Press/Pearson, ISBN 978-0-13-3505980-9).    Written by five IBM data management professionals, this book offers up new ways to integrate social, mobile, location, and traditional data.

Even with all of the new big data and analytics books being published this book is worth seeking out for its unique perspective and focus. Covering IBM’s experience with using social MDM at enterprise customer sites, the book provides guidance on improving relationships, enhancing prospect targeting, and fully engaging with customers.

In Beyond Big Data you will be introduced to a not only the basic concepts of master data and MDM, but its role with social data. By combining social and master data the book shows how to derive insight from this data and to incorporate it into your business.

Along the way the authors help to explain how social MDM extends fundamental MDM concepts and techniques, method of architecting a social MDM platform, using social MDM to identify high-value relationships and more. The book even tackles thorny issues like the ethics and privacy concerns of gathering and using social MDM.

What the book is not, is another tome attempting to describe Big Data; what is, is a useful approach and roadmap to exploiting a new Big Data niche – social MDM.

And last, but definitely not least, we have Rick Sherman’s impressive new book Business Intelligence Guidebook: From Data Integration to Analytics (2015, Morgan Kaufmann, ISBN 978-0-12-411461-6).

This 500+ page tome highlights the importance of data to the modern business and describes how to exploit data to gain business insight. Throughout the course 19 chapters describes the requirements, architecture, design, and practices that should be used to build business intelligence applications. The pratical advice and guidelines offered within the pages of Sherman’s book will help you to build successful BI, DW and data integration solutions.

The overarching theme that business people need to participate in the entire process of building business intelligence application is incorporated into the entire book. Each of the seven (7) parts that the book is organized into provides useful and actionable knowledge covering the following areas:

  • Concepts and Context
  • Business and Technical Needs
  • Architectural Framework
  • Data Design
  • Data Integration Design
  • Business Intelligence Design
  • Organization

By deploying the information contained in this guidebook, you should be able to successfully justify, launch, develop, manage and deliver a highly-functioning business intelligence system. And do it on time and within budget.

A companion website includes templates and examples, further discussion of key topics, instructor materials, and references to trusted industry sources.

All in all, we have three recommended new data books that are well worth your time to seek out and read. Doing so can only help to improve your data knowledge and employabaility.

Other Books of Note

Posted in DBA | Tagged , , , | 1 Comment

A Couple of Database Predictions

I usually try to avoid predicting the future because humans, as a general rule, are not very good at it. But when things look repetitive or cyclical, then sometimes a “bold” prediction here and there is worth attempting.

So, with that in mind, over the course of the next 5 years or so I predict widespread consolidation in the NoSQL DBMS market. As of today (February 18, 2015), NoSQL-Database.org lists 150 different NoSQL database systems. Anybody should be able to foresee that a number of NoSQL offerings so vast is not sustainable for very long. Winners will emerge pushing laggards out of business (or to languish).

Why are there so many NoSQL options? Well, IT folks like options. Many did not necessarily want to be tied to the Big Three RDBMS vendors (Oracle, IBM and Microsoft); others were looking for novel ways to solve problems that were not very well served by relational offerings. But nobody wants a sea of incompatible, proprietary DBMSes for very long because it is hard to support, hard to train talent for, and hard to find new employees with experience.

So consolidation will happen.

Additionally, the Big Three RDBMS vendors (and others) will work to incorporate the best features and functionality of the NoSQL database systems into their products. This is already happening (witness the column store capability of IBM DB2 for LUW with BLU Acceleration). We will almost certainly see more NoSQL capabilities being added to the relational world.

So there you have it: two “bold” predictions.

  1. The NoSQL database market will consolidate with a few winners, some middle of the packers, and many losers.
  2. The relational database market will combat NoSQL by adding capabilities to their existing portfolio

What do you think? Do these predictions make sense to you? Add your thoughts and comments below…

Posted in NoSQL | 2 Comments

Models Should Form the Basis of Database and Application Development

One of the biggest challenges facing today’s organizations is ensuring the right information reaches the right people so data is used appropriately and accurately. In the age of Big Data and analytics, there is even more data being churned out more frequently than ever before. And few organizations treat data as the corporate asset it truly is, so documentation is sparse to non-existent.

Data architects can design the best possible data model, but it does not necessarily follow that developers will use the information how it was intended, or that business analysts will immediately understand the relationships between different data elements.

An integrated solution for modeling can help reduce this problem by streamlining information exchange and contextualizing the data model to minimize errors. By using models as the common language for all data stakeholders–-whether application or data warehouse developers, Business Intelligence (BI) professionals, data scientists or business users–-organizations can effectively translate otherwise disparate requirements in context of one another. For example, a data architect passing a logical data model to a development DBA needs to ensure the design is properly implemented in the database so the business’s rules are enforced. With an integrated approach, models–-and the underlying metadata–-are smoothly and accurately passed across functional boundaries.

The models for the data architect and the DBA remain similar in look and feel, and utilize the same types of notation. Deploying modeling solutions with tight integration across functional areas can reduce the amount of time, effort, and error involved in sharing information. And sharing at the model level simplifies potentially complex processes because models can more readily be consumed and understood by all required parties.

In this context, a model isn’t just a diagram with boxes and arrows. Yes, a data model must include the visual representation of the data (such as an entity/relationship diagram), but it also must be backed up by metadata. A model without metadata might as well be just a quick picture scrawled on a napkin somewhere. It’s the powerful combination of a clear diagram and the metadata definitions for the elements in the diagram that make a model useful.

Facilitating Data Integration

Among other objectives, a good data modeling solution can help you reverse engineer existing schemata into models and forward engineer models into new target schemata. This is basically what you do during an ETL process: You reverse engineer an existing data source (you extract the data along with the accompanying metadata, and it’s the latter that’s described in the schema), transform that data, and then forward engineer the target data.

For this reason, it makes sense to utilize models as a mechanism to facilitate data movement and data warehouse population. A model-driven approach to data integration can enable data architects and BI professionals to import data models and the associated metadata for use during ETL. This visual approach to data integration helps simplify the process while ensuring the complete and accurate migration and integration of data between systems.

Simplifying Application Development

Models can also be used to guide application development. The comprehensive data model can be integrated into a process model to streamline the development process. With knowledge of and access to the metadata, developers are better prepared to build the correct processes the first time.

Additionally, application modeling solutions used by development teams enable clear application models to be created that can then be translated into code. Model-based application development enables the deployment of higher quality applications in less time. And with the appropriate tools, you can then generate code directly from models into popular application development technologies.

Data Modeling, NoSQL and Big Data

It is sometimes said that Big Data and NoSQL database implementations benefit from schema flexibility. This means that one record may contain different data elements than another record within the same database row (or object or document or…) This would seem to obviate the need for a data model.

But it should not! If you simply determine that the developer or user can add any data at any time anywhere in the database then you will end up with a pile of mismatched, unmanaged data – or what I like to call a data outhouse. So sure, the flexibility offered in the NoSQL world can be a useful tool, but without a data model it turns into a disaster.

NoSQL database systems grew out of the need to resolve certain big data management issues. But running systems without a definition of the data being managed should not be the result. If it is, then things will go from good to bad to worse!

Enhancing Data Integrity Across the Enterprise

Creating data models and collecting metadata that spans the breadth of an organization can deliver significant benefits such as better decision support and business information, or richer customer information and support. However, these benefits are realized only when the information is accessible and accurate. Using integrated modeling solutions to implement model-driven data management and development can help organizations share knowledge effectively and with minimum effort. And it will certainly produce higher quality applications and data, too.

A logical data model should be used as the blueprint for designing and creating physical databases. Modeling can be integrated into your IT infrastructure, thereby helping to exploit the synergy that exists within your databases, IT activities, and application systems.

Posted in analytics, Big Data, data modeling, NoSQL | 1 Comment

2014 in review

The WordPress.com stats helper monkeys prepared a 2014 annual report for this blog.

Here’s an excerpt:

The concert hall at the Sydney Opera House holds 2,700 people. This blog was viewed about 42,000 times in 2014. If it were a concert at Sydney Opera House, it would take about 16 sold-out performances for that many people to see it.

Click here to see the complete report.

Posted in DBA | 1 Comment

Happy Holidays

Cheers!

happy-holidays

Just a short post to end the year wishing all of my readers everywhere a very happy holiday season! Enjoy the holidays and come back next year (2015) as we continue to explore the world of data and database technology…

 

Posted in DBA | Leave a comment

I’m On Toad World… and the datAvail Blog, too!

Just a short post today to let folks know that my DB2portal blog is being syndicated to the Dell Toad World portal. That means that you can follow all of my DB2-specific blog posts either at DB2portal or at Toad World.

If you follow me at DB2portal – and I do want to encourage you to continue to do that – be sure to periodically check in on my posts at Toad World, too. That is because I write a separate, exclusive post monthly for the DB2 channel on Toad World. The latest exclusive Toad World post is titled Controlling the Rolling Four Hour Average and DB2 Costs.

Additional recent posts include:

And I’d like to alert you to another new outpost for my DB2-related material at the datAvail Blog. You can think of the posts there as being the “Best of DB2 Portal.” I will be reviving some of my past content and updating it so that it is accurate and current. My most recent post there is DB2 Compression: z/OS versus LUW.

Yes, I am keeping quite busy writing about DB2, data and database issues – but I enjoy writing, so I’ll keep it up. Sorry if it is confusing with all of the outlets for my material. But here is a good rule of thumb:

That’s all for today!

Posted in data, DB2 | 1 Comment

A Data Quality Initiative

As regular readers of this blog know, I sometimes share answers to questions I get via e-mail (and other avenues). By doing so I can hopefully expand on the audience participating in the discussion…

Anyway, I received the following question via e-mail:

When trying to get management on board with a data quality initiative, what should I focus on in order to justify the investment? How do I build “the business case?”

This was my response:

The best (short) answer that I can to this question is to try to quantify the cost of poor quality data on the business. This needs to be written in business language and not technology-speak. For example, what is the cost of a lost sale because product information was incorrect? Do you have a way to identify these cases? Even anecdotal evidence can be powerful if you are talking to the manager of the product line that lost the sale.

A more in-depth example might be an approved (or unapproved) business decision made based upon the wrong data and assumption. What is the cost, in terms of lost business? Or a hiring decision made based on erroneous information. What is the cost, in terms of an ineffective work team (plus the cost of removing and replacing the bad hire).

The cost of poor data quality is pervasive… and it can be difficult to quantify effectively.

I realize that the cost of finding these problems can be enormous, too. It can help to have some industry expert help. I would recommend that you purchase and read any of the several excellent books that Thomas C. Redman has written. These books focus on the data quality problems and have some facts and figures on average cost of poor quality data to business. For more in-depth and technical treatments of data quality issues I would direct you to books written by Jack Olson and Larry English.

…I realize that this is a quick and dirty answer to a complex question, but that is usually all I can afford to do with e-mail Q+As. Did I get the basics covered? What would you have answered?

Please share your thoughts and comments…

Posted in Data Quality | Leave a comment

Database Consistency Models

Just a short blog post today to point folks to a very well-written article on database consistency models titled On the Futility of Custom Consistency Models (posted on the Hacking, Distributed blog by 

This blog post does a very nice job of discussing the current trend of more and varied consistency models for database systems. The general idea seems to be that you can eliminate some of the overhead of ensuring data consistency for some applications. And, sure, there is some validity there. But how many different consistency models do we really need?

Also, the notion that relational/SQL DBMS products do not allow for flexible consistency is absurd. For example, I’ve written about isolation levels in this blog before, and you can adjust how locking behaves — and therefore the consistency of query results — by adjusting the isolation level. For example, an uncommited read (or “dirty” read”) can be used to eliminate read locks in DB2, and therefore return dirty data. Applications using dirty reads are more efficient, but they might return bad data to the application. For some use cases this might be fine, but I sure wouldn’t want my bank to use dirty reads on my financial transactions!

So the next time you start reading about eventual consistency and how it is revolutionary, step back, reconsider, and think about what you are reading. There is merit to it for some use cases (e.g. my seller account on amazon.com), but not for most (e.g. when the data absolutely must be accurate every time you read it).

Posted in data, data integrity, Data Quality, database design | 1 Comment