When you get certificates included in a different keystore type like it is the case of PKCS12 (commonly using the *.p12 extension) you need to extract and add them to the JKS keystore. Failure to do so will make your program depending on individual keystore files rather than just one keystore where all certificates and private keys are kept.
Here is how you can do it (sample using OSX paths but applicable to other OS as well).
First find the key for the certificate to export. Basically the left first word for the specific entry from this command:
keytool -list -keystore /Users/nestor/Downloads/cert.p12 -storetype pkcs12Then run something like the below. Note that alias.from.cert.p12 comes from the previous command.
keytool -importkeystore -srckeystore /Users/nestor/Downloads/cert.p12 -destkeystore /Library/Java/Home/lib/security/cacerts -srcstoretype PKCS12 -deststoretype JKS -srcstorepass cert.p12.password -deststorepass changeit.is.the.default.password -srcalias alias.from.cert.p12 -destalias alias.for.cacerts.new.certificate
Note that the private key inside PKCS12 might need a password and that password must be the same as the JKS where you are importing. Failure to do this will end up in Access Denied, Forbidden or any other error comming from the Server where the key is attempted to be used.