Decoupling Secret Management from Applications and process — Secret as a Service
The Secret Management or Secret service is a method of managing secrets and sensitive part of the IT. The secrets can be 1. User & Application Credentials 2. API secrets 3. Data encryption 4.Machine Identity 5.Devops process 6.Container information etc.
What is the problem of existing process?
If you store the secrets in files or application source, you are inviting many problems like
- Easy expose of secrets to unknowns
- Vulnerability instances
- Information’s are stored in many places which becomes maintenance and governance bottlenecks
- Passed secrets around clear text across application layers and applications in an Enterprise etc.
Most of the application uses anti-pattern like email, Git Hub, hard coding, config files & file systems to use for storage or transferring the information between systems, environments and parties.
Recommend is to decouple Secret Management from source, config, data to made externalize methods to manage these secrets in an enterprise.
You can always argue like, we do not come across such instances or don’t find better use case to externalize secret management.
Why we need Secret Management?
Let’s take couple of use cases where secret management used.
- Data protection is a top priority in any organization, which means that the database credential rotation is a critical part of any data protection method. Each role in an application has a set of different permissions granted to access the database, assume when hackers attacks your system, how will you handle with the fixed credentials and roles, you can avoid maximum by providing continuous credential rotation.
How will secret management helps to resolve about scenario:
Application request secret management services for database credentials rather than setting them as part of the environment variables or application config files. The admin team specifies the TTL of the database credentials to enforce its validity so that they are automatically revoked when they are no longer used.
Each application will get unique credentials that no need to share across. By making the TTL short lived, you reduced the change of the secret being compromised.
Lets consider the DevOps process, most of our build environments are insecure, DevOps engineer and software engineer feel security during the process slow them down, so they often bypass security controls in development process. Build environment normally with devops engineer control, so few outsiders know where they are or how they operate. As steadily increasing automation to improve agility, things speed up and we remove human intervention. Continuous Integration and deployment use automation to speed up delivery and Jenkins and maven automatically regenerate application code, formation templates, configuration files are checked into Artifactory. When build completes, we automatically launch new environments to perform test. When these test pass, some organizations launch the code in production. In this case, the server security is very important topic and we no longer have the luxury of leaving passwords, encryption keys and access tokens sitting in unprotected file or scripts.
As continuous integration improve agility, we need to automate the provisioning of secrets into the process as well, if you integrate secret management tool to DevOps process, it will improve the process and speed.
As detailed in use cases and below reasons, I will suggest to have separate secret management
- Due to increase in complexity in system and business
- Due to emergence of distributed technologies and real time
- Change in software development and deployment
- Emergence of lot of technological options
- Because of more cyber attacks and vulnerability from both in and out of enterprise
- Many mainstream enterprises are interested to streamline secret management
you can always argue that the traditional HSMs and encryption and storage management are good to store information, You can still continue to use but as mentioned earlier, these storage mechanism is legacy way of handing and might encounter difficulty to use along with Devops, Pem files, Microservices secrets, tokens etc., for with the existing technology might not have capability to address to all. We can continue to use but radar.
Last few months I was interacting with Microservice Engineer, DevOps Engineer on problem they are facing on managing the secrets, I encountered some few concerns as detailed below:
- Secrets are managed in configuration file and sometime exchange of secrets over email
- Focusses more on encryption, secret at rest and secret at transit not really on API keys and managed with encryption
- Traditional approach not work well with the latest DevOps and agility.
- Not aligned with Microservices with Docker and Kubernetes architecture
How to solve these problems?
There are set of good software and tools available in Industry that manages secrets and exposes either API services or Fuse file system.
With this approach, rather than hard wiring or configuring secrets as part of environment, applications retrieve from a separate process.
These third-party software’s provides
- Secure secret storage, Dynamic secrets, Revocation, Versioning
- Easy way of management with UI/UX along with APIs
- Integrates with any software system
- Audit management
- High Availability and scalability
Who are all providing Secret Management as a Service?
There are open source stacks available to manage secrets, below list are few
- Hashicorp Vault — It is a tool for accessing secrets and provide a unified interfaces to any secrets & expose secrets via secure APIs, more detail visit (https://www.vaultproject.io/)
- Conjur — It provides automated privileged account servicing and secret management and platform to secure devops lifecycle and cloud native environment (https://www.conjur.org/)
- Keywhiz — This software has feature to store and manage applications and infrastructure secrets (https://square.github.io/keywhiz/)
- Knox — This software has feature to store confidential secret keys, key relation, audit log report to track of systems and user access confidential data (https://github.com/pinterest/knox)
- Confidant — It provides credentials to internal and external services, SSL keys etc. these services works with multi-environment & manage version management. This tool uses Dynamo DB to store secrets & exposes details via secured APIs (https://github.com/lyft/confidant)
- Docker Secrets — This tool provides a container native secret solution that strengthen the trusted delivery component of container secrets (https://blog.docker.com/2017/02/docker-secrets-management/)
- T-Vault — This service from T-Mobile and built on top of Hashicorp Vault (https://opensource.t-mobile.com/blog/posts/introducing-tvault/)
- Torus — It provide secret management on hosted approach rather than API approach (https://www.torus.sh/)
- Creed Stash — It is a distributed credential management system and built on top of AWS KMS system.
- Sneaker — It is a utility for storing sensitive information by using AWS KMS and S3 (https://github.com/codahale/sneaker)
- AWS Secret Manager — It is a tool enables AWS uses to manage secrets & credentials instead saving them on disk (https://aws.amazon.com/secrets-manager/) and you can watch video (https://www.youtube.com/watch?v=Y3Gn_iP3FlE&feature=youtu.be)
- Azure Key Vault — It provides a services that enables to manage all secrets in one place (https://azure.microsoft.com/en-us/services/key-vault/) and you can watch video (https://www.youtube.com/watch?v=fVSYaFhMA5Y&feature=youtu.be)
- Unbound Tech — It is a multi-party computation for key management (https://www.unboundtech.com/)
What are the challenges of Secret Management tools?
Challenges of Secret Management tools are based on how it supports for enterprise application with lots of disruption in technology and business. I detailed some of the challenges, these not limited to just below.
- How secret management works with Multi-cloud environment?
- How these tools supports scalability, High Availability and Resilience?
- How these tools integrate with the existing key system and IAM?
- How these tools provide UI/UX to manage secrets?
- Who owns these tools in an enterprise?
- Who are the users of the secret management tools?
- How to access these tools, is it API or file base?
- How these tools support dynamic changes in real time?
- How these systems support auditing etc.?