Added rough version of use of Data protection API and saving key protected to file

[geertendoornenbal]

I've added some code to show how to use the data protection API, and save the protected key to file.

However, it is a rough version, and I'm wondering if saving to file is the most secure option.

I also removed the RSAKeyParametersWithPrivate class, because it is not necessary anymore.

Please note that I quickly threw this together from my own project, to show you how this can be done.

[dazinator]

I stumbled accross this, and thought i'd give it a whirl. It looks like I need to create the key file first though as if I just run this application as is I see this:

Any ideas what I need to do to create a key in the first place?

[geertendoornenbal]

You can create a simple console project that calls the function GenerateProtectedKeyToFile to create a key file.
Or change the startup temporarily with the creation of a file, and change it back to load it.

After that you should be able to run the application as it is.

[dazinator]

Thanks, I have tried calling GenerateProtectedKeyToFile - and it created the following, but it doesn't appear to work.

Because it falls over Deserializing the KeyParams from the authtoken.key file - becuase the file text isn't JSON - instead the file text has actually been encrypted and needs decrypting first:

 public RSAParameters GetKeyParameters(string file)
        {
            if (!File.Exists(file)) throw new FileNotFoundException("Check configuration - cannot find auth key file: " + file);          

            var keyParams = JsonConvert.DeserializeObject<RSAParameters>(File.ReadAllText(file));
            return keyParams;
        }

The above doesn't work because when authtoken.key is written, the JSON is first Protect()'ed before being written, as per below:

        private void GenerateKeyAndSave(string file)
        {
            var lRandomKey = GetRandomKey();
            string lSerializedParameters = JsonConvert.SerializeObject(lRandomKey);

            string lProtectedString = mProtector.Protect(lSerializedParameters);
            File.WriteAllText(file, lProtectedString);
        }

Therefore I tried chainging the load key params method to do an Unprotect first but that doesn't work either - it falls over on the call to Unprotect..

public RSAParameters GetKeyParameters(string file)
        {
         if (!File.Exists(file)) throw new FileNotFoundException("Check configuration - cannot find auth key file: " + file);
            var text = File.ReadAllText(file);           
            var unprotectedString = mProtector.Unprotect(text);

            var keyParams = JsonConvert.DeserializeObject<RSAParameters>(text);
            return keyParams;
        }

So basically, struggling to see how this all works at present!

[dazinator]

Oh, also in case it helps, when I try Unprotecting() the protected file contents (authtoken.key) - the exception I get is the following:

So perhaps I am on the right track but Data Protection needs to be configured differently or something?

[geertendoornenbal]

Did you create one from a different application? If so you need to specify an appname, to make it come from the 'same' source. This is mentioned in this line of code:
// uncomment when doing this from different application
//configure.SetApplicationName("SameAppName");

[geertendoornenbal]

So the creation has to be done like this:

var lServiceCollection = new ServiceCollection();
lServiceCollection.AddDataProtection(configure =>
{
  configure.SetApplicationName("appname");
});
var lServices = lServiceCollection.BuildServiceProvider();

// create an instance of MyClass using the service provider
var lKeyUtils = ActivatorUtilities.CreateInstance<clsRSAKeyUtils>(lServices);

lKeyUtils.GenerateKeyAndSave("../dir", lServices.GetRequiredService<IRuntimeEnvironment>());

[dazinator]

No I am creating within the same application. I think I have figured out the issues and managed to get it to work but I had to make code changes / fixes. The two problems are:

1. It protects the json string using dataprotection, before writing the authtoken.key file, but when reading the authtoken.key file it doesn't attempt to unprotect the file contents to get back the original JSON before deserilising. Perhaps i'll add line comments to make this clearer. This must be a bug as I fail to see how this can possibly work at present.
2. I added in the Unprotect() call but Data Protection complained saying it couldn;t unprotect! I managed to solve this issue by getting rid of the static methods, as I think the creation of the seperate instances of the data protection provider may have been to blame.

[dazinator]

Here - the json is being protected before being written to the file..

[geertendoornenbal]

You're right. In the project where I had this set up, I did the same things you mentioned (unprotect, and made IDataProtector a member of the RSAKeyUtils class, with dependency injection).

Apparently I forgot to move this to the example code.

[dazinator]

No problems, glad I am not going crazy! I have it working now so it was still valuable as sample code ;)

[geertendoornenbal]

Good to hear you have it working! I added this in a rush, because I spend quite some time on it to get it to work, and wanted to provide the code so others could save time ;)

[geertendoornenbal]

Just to be complete: I added the DataProtector as follows:

IDataProtector mProtector;

public RSAKeyUtils(IDataProtectionProvider aProvider)
{
   mProtector = aProvider.CreateProtector("appname");
}

public static TokenAuthOptions GetTokenOptions(IServiceProvider aServiceProvider)
    {
      RSAKeyUtils lRsaKeyUtils = ActivatorUtilities.CreateInstance<RSAKeyUtils>(aServiceProvider);
      // Create the key, and a set of token options to record signing credentials 
      // using that key, along with the other parameters we will need in the 
      // token controlller.
      RSAParameters lKeyParams = lRsaKeyUtils.GetKeyParameters(cTokenKeyFile);
      RsaSecurityKey lKey = new RsaSecurityKey(lKeyParams);
      TokenAuthOptions lTokenOptions = new TokenAuthOptions()
      {
        Audience = clsRSAKeyUtils.cTokenAudience,
        Issuer = clsRSAKeyUtils.cTokenIssuer,
        SigningCredentials = new SigningCredentials(lKey, SecurityAlgorithms.RsaSha256Signature)
      };
      return lTokenOptions;
    }

And use it as follows:

      aServices.AddDataProtection(configure =>
      {
        configure.SetApplicationName("appname");
      });
      var lServices = aServices.BuildServiceProvider();

      mTokenOptions = RSAKeyUtils.GetTokenOptions(lServices);

[dazinator]

Cool. I'm wondering if, now that I have an "authtoken.key" file - will DataProetection be able to Unprotect() and read that file forever more? I know that DataProtection can do key rotation? Or perhaps does this automatically, and I am just wondering if in X days time, all of a sudden DataProtection will start using different keys, and and will no longer be able to Unprotect the contents of the authtoken.key file?

[geertendoornenbal]

I've wondered the same thing. It's hard to figure this out based on only the documentation (http://docs.asp.net/en/latest/security/data-protection/, especially look at encryption at rest). I could not really figure out what really was going on and what the best approach was.

[mrsheepuk]

Indeed - this worry (regarding what happens when the DPAPI rolls the keys over) is why I haven't merged this pull request in yet, I want to know the answer to that, else every time the keys get rolled, all the authentication tokens will become unreadable... As I understand it, the old keys should remain available for unprotecting for a certain amount of time, even once new keys have been generated by DPAPI, but unless the key file gets re-written by something, sooner or later it seems that it's going to expire.

[dazinator]

Yeah.. my first thought was to put a simple try catch around the method that Unprotects()'s and reads the key. If it fails to Unprotect() (because the DPAPI keys have just changed) then delete the file and generate a new key file, protected with latest DPAPI, then read that one and return the result. I think this would work in that now tokens will be signed with a new key, but made me wonder what happens tot he existing tokens sitting in people's browsers etc when this happens.. Assuming they'd all now be invalid.. So now I am wondering how to make it so that everyone doesn't get thrown out of the system when this process happens! :(

[dazinator]

Just trying to think of the reasons why you'd generate a new key file:

1. Your existing key was compromised in some way, so you needed to "reset" security.
2. You physically cannot read the old key anymore, perhaps you accidentally destroyed it, (or DPAPI keys changed if you use that to protect it)

I can understand 1) which is a rare event and so kicking people out of the system and asking them to re-authenticate isn't a huge deal in a rare scenario like that.

But as for 2) - DPAPI forcing you to change the key file when it rolls its keys seems like not a good idea. I think you'd almost want to guarantee that you can read the key (using DPAPI or whatever protection mechanism) for the lifetime of the key, and you don;t want DPAPI or whatever to dictate that lifetime.. as it has massive impact on your users potentially.

[dazinator]

Because of the above, I don't think DPAPI makes sense unless you can configure it in such a way that you are guaranteed to be able to read the key file for the lifetime of the key. Time to hit the DPAPI docs and see if that's possible! :)

[mrsheepuk]

Indeed... it might "just work", in that if it's configured correctly, it might be the case that the old expired keys are ALWAYS available for unprotect operations. But I read the docs over and over and couldn't really convince myself that this was what happened, but also couldn't think how to test it easily without waiting 45 days!

[dazinator]

@mrsheepuk haha I see.. I'll see what I can find out

[dazinator]

Feel free to chime in: aspnet/DataProtection#137

[mrsheepuk]

I've just had another look at the documentation, it has been updated since I last looked - it now reads:

... there is nothing prohibiting a developer from using the ASP.NET 5 data protection APIs for long-term protection of confidential data. Keys are never removed from the key ring, so IDataProtector.Unprotect can always recover existing payloads as long as the keys are available and valid.

This would imply that it is always safe to use this method, so long as the DPAPI is configured to keep the keys in a way which is accessible to all machines in a cluster of web servers.

[geertendoornenbal]

And as long your key won't be compromised (and you know about it!) ;)

[dazinator]

Ah excellent news ;)

[mrsheepuk]

It feels like there's still an extra hop than should be needed. It would be ideal to use the DPAPI directly to sign and verify the tokens, thus removing the entire need to generate and store a key file on disk. If we could find a way to hook the "unprotect" into the JWT verify token stage... Then it would be as simple as configure the DPAPI and off you go!

I'll have another play...

[dazinator]

@mrsheepuk Yep - I completely agree!

[Icestorm0141]

@mrsheepuk not sure this would work if you need to share the public key across multiple API servers. You'd need a file written out to disk somewhere ahead of time to be shared with the API servers. Generating the key's on the fly doesnt really work, unless somehow you pulled in a database/key store into the mix. I dont even want to go there.

Correct me if I'm wrong please. I'm new to this land of security.

[mrsheepuk]

Hi @Icestorm0141 - from the DP API documentation you can choose to store the keys on a UNC path accessible by all machines which need them. The DPAPI then manages adding new keys to the shared storage and timing out old keys. To use a UNC path, it sounds like you need X.509 certificates set up for all machines which are used to secure the keys at rest, I think.

I'm not 100% clear on this...

[zhouhao27]

Why this pull request not merged into main branch?