First, a disclosure: This article is a modified version of an introduction commissioned for a vendor white paper. However, I feel that redaction is important for a larger audience, this piece is a general piece, and vendors are touched upon only in the "Competitive Landscape" section.
Understand first that the complexities in the "data explosion era" are not only about storing more data, but also about managing its use. Critical to proper information management is not only ensuring that only the right people have access to that information, but that they are able to use it for legitimate purposes. Potential conflicts exist because ensuring the availability and openness of information to which some are entitled necessitates restrictions preventing access others who are not entitled because of privacy and confidentiality requirements.
Take two illustrations from the U.S. Government: The intent of the Freedom of Information Act (FOIA) is to make government organizations more accountable for their actions by making information about those actions more available on demand. On the other hand, the U.S. government's Health Insurance Portability and Accountability Act (HIPAA) is designed to prevent the unauthorized disclosure of individuals' personal healthcare information. However, though FOIA is about openness, individuals are not entitled to access sensitive personal information or national security information embedded in documents that could otherwise be made publicly available. On the HIPAA side, physicians or other healthcare professionals with a legitimate need to know should not be prevented from accessing electronic health records for valid and beneficial purposes, but do not need to see billing information that is unrelated to providing care for patients.
So what can technology do to enable or enhance the complex balancing act between openness and privacy? The first line of defense consists of access controls that manage who is entitled to see what information. One possible solution is in the use of data loss prevention (DLP) software, which can restrict the transmission (such as through e-mail) of sensitive information from one authorized user to another unauthorized user. Encryption is also an important means for ensuring data confidentiality. But these approaches, with some exceptions, tend to be blunt instruments which too often restrict access more than is really necessary. For example, DLP solutions can reject the transmission of an email if a document attached to it contains a name associated with a social security number. That may sound sensible, but what if the intended recipient needs other information contained in that document for legitimate purposes and has no interest in or need for just the name and social security number content.