-
Notifications
You must be signed in to change notification settings - Fork 937
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Provide a way to dynamically update TLS certificates #5228
Conversation
Hi, @ikhoon ! |
core/src/main/java/com/linecorp/armeria/client/HttpClientFactory.java
Outdated
Show resolved
Hide resolved
core/src/main/java/com/linecorp/armeria/client/TlsProviderBuilder.java
Outdated
Show resolved
Hide resolved
core/src/main/java/com/linecorp/armeria/client/TlsProviderBuilder.java
Outdated
Show resolved
Hide resolved
The priority of this issue has been raised. Please let me finish this PR. 🙇♂️ |
I'm sorry to be late for this reply. |
No worries. I’m really grateful for your contribution. 🙇♂️ |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #5228 +/- ##
============================================
- Coverage 74.75% 73.62% -1.14%
+ Complexity 21409 20168 -1241
============================================
Files 1877 1748 -129
Lines 79126 74634 -4492
Branches 10201 9526 -675
============================================
- Hits 59152 54946 -4206
+ Misses 15179 15139 -40
+ Partials 4795 4549 -246 ☔ View full report in Codecov by Sentry. |
This PR is ready to review. PTAL. The PR description has also been updated for the API design and implementation details. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall looks good! Server side looks good to me 👍 Left some minor questions on the client side 🙇
core/src/main/java/com/linecorp/armeria/common/metric/EventLoopMetrics.java
Show resolved
Hide resolved
core/src/main/java/com/linecorp/armeria/client/HttpChannelPool.java
Outdated
Show resolved
Hide resolved
core/src/main/java/com/linecorp/armeria/client/HttpChannelPool.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks promising! 👍
core/src/main/java/com/linecorp/armeria/common/metric/EventLoopMetrics.java
Show resolved
Hide resolved
core/src/main/java/com/linecorp/armeria/client/ClientFactoryBuilder.java
Outdated
Show resolved
Hide resolved
core/src/main/java/com/linecorp/armeria/client/ClientFactoryBuilder.java
Show resolved
Hide resolved
core/src/main/java/com/linecorp/armeria/common/AbstractTlsProviderBuilder.java
Outdated
Show resolved
Hide resolved
core/src/main/java/com/linecorp/armeria/client/ClientFactoryBuilder.java
Outdated
Show resolved
Hide resolved
@Deprecated | ||
default boolean allowsUnsafeCiphers() { | ||
return false; | ||
} | ||
|
||
/** | ||
* Returns the {@link Consumer} which can arbitrarily configure the {@link SslContextBuilder} that will be | ||
* applied to the SSL session. | ||
*/ | ||
default Consumer<SslContextBuilder> tlsCustomizer() { | ||
return builder -> {}; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That will move the responsibility of managing the life cycle of SslContext to TlsProvider, which may complicate things.
What do you think of modifying the method for creating the SSL context so that the TlsProvider
doesn't have to manage the lifecycle at all? Here's a proposed interface change:
interface TlsProvider {
SslContext createSslContext(String hostname, SessionProtocol sessionProtocol, TlsEngineType tlsEngineType);
}
I'm suggesting this because it seems like the current TlsProvider
functions more as a data class than as a provider.
TlsProvider is designed to be used at both client and server.
At present, we can use TlsProvider.ofServer()
for the client side as well. By validating the TlsProvider
when it's assigned to a client or a server, we can prevent this misuse.
If you still prefer to use TlsProvider
for both, we could add enum parameters to TlsEngineType
such as CLIENT_JDK
, SERVER_JDK
, CLIENT_OPENSSL
, etc.
Users who implement their own TlsProvider may find it difficult to build SslContext, or we might need to expose a factory method to make it easier.
That is true. I also worry about exposing Netty's SslContext API to the public. However, creating an SslContext might not be too difficult since Netty provides a well-designed SslContextBuilder API. To address the issue of exposing the API, we could provide our own abstraction, similar to what we did with DnsEndpointGroupBuilder
.
core/src/main/java/com/linecorp/armeria/client/ClientFactoryBuilder.java
Show resolved
Hide resolved
This PR is ready to review again. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great! 👍 👍 👍 Left minor comments and questions. 🙇
core/src/main/java/com/linecorp/armeria/client/ClientTlsConfigBuilder.java
Outdated
Show resolved
Hide resolved
core/src/main/java/com/linecorp/armeria/common/TlsProvider.java
Outdated
Show resolved
Hide resolved
core/src/main/java/com/linecorp/armeria/common/TlsProviderBuilder.java
Outdated
Show resolved
Hide resolved
core/src/main/java/com/linecorp/armeria/server/ServerBuilder.java
Outdated
Show resolved
Hide resolved
core/src/main/java/com/linecorp/armeria/internal/common/SslContextFactory.java
Show resolved
Hide resolved
|
||
final Bootstrap baseBootstrap = isDomainSocket ? unixBaseBootstrap : inetBaseBootstrap; | ||
assert baseBootstrap != null; | ||
return newBootstrap(baseBootstrap, remoteAddress, desiredProtocol, serializationFormat); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So, we should create bootstrap every time when connecting if sslContextFactory
is set?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right. Because we need a new ChannelInitializer
for a different SslContext
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If so, I'm a bit worried about the performance. 🤔
Maybe @trustin can give some idea.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When I saw Bootstrap
, I thought it was a simple class and did not refer to heavy objects.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! 🙇 🙇 🙇
core/src/main/java/com/linecorp/armeria/common/metric/CertificateMetrics.java
Outdated
Show resolved
Hide resolved
core/src/main/java/com/linecorp/armeria/client/ClientFactoryBuilder.java
Show resolved
Hide resolved
final List<X509Certificate> trustedCerts = tlsProvider.trustedCertificates(); | ||
if (trustedCerts != null && !trustedCerts.isEmpty()) { | ||
contextBuilder.trustManager(trustedCerts); | ||
} | ||
applyTlsConfig(contextBuilder); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The fact that tlsProvider.trustedCertificates
can be overwritten by clientTlsConfig.tlsNoVerifySet()
or clientTlsConfig.insecureHosts()
was surprising to me.
Is this intentional?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we have the same behavior before. Trusted certificates set with ClientFactoryBuilder.tlsCustomizer(c -> c.trustManager(certs))
got overwritten by ClientFactoryBuilder.tlsNoVerifySet()
.
tlsProvider.trustedCertificates
and clientTlsConfig.tlsNoVerifySet()
or clientTlsConfig.insecureHosts()
are mutually exclusive. If we respect trustedCertificates
, tlsNoVerifySet
is also ignored.
What do you think of adding a note for the behavior?
* Provides client-side TLS configuration for {@link TlsProvider}. | ||
*/ | ||
@UnstableApi | ||
public final class ClientTlsConfig extends AbstractTlsConfig { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For the record, can you explain what properties go into KeyPair
and what properties go into TlsProvider
?
i.e. tlsNoVerifySet
, insecureHosts
seems to modify the trusted certificates, yet they are in ClientTlsConfig
. On the other hand, TlsProvider
has a trustedCertificates
method
I'm wondering if it's just easier to expose an interface which is a simplified version of TrustManager
(similarly to how KeyPair
is a pojo version of KeyManager
)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did you imagine to return the simplified TrustManager
instead of List<X509Certificate>
for trustedCertificates()
?
I hesitated to add the simplified TrustManager
because TlsProvider
was designed to use both server and client but tlsNoVerifySet
and insecureHosts
would be only useful for clients. Moreover, they are not a recommended option for production.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did you imagine to return the simplified TrustManager instead of List for trustedCertificates()?
Right, I imagined that TlsProvider
contains a trustManager()
-esque API
I hesitated to add the simplified TrustManager because TlsProvider was designed to use both server and client but tlsNoVerifySet and insecureHosts would be only useful for clients. Moreover, they are not a recommended option for production.
I thought TrustManager
is used by both server and client side.
I still find it odd that TrustManager
related APIs exist in both APIs (TlsProvider
, ClientTlsConfig
), but I'd rather not delay this PR
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think TlsProvider's function is to provide the data required for TLS handshake rather than associating with TrustedManger
.
tlsNoVerifySet
, insecureHosts
are options that exist at the code level so it would be ambiguous to express them as data. I thought ClientTlsConfig
was a better place to set them.
I understand your concerns and this API may cause confusion or inconvenience. Let's get user feedback and decide whether the current design needs refactoring.
final TlsKeyPair tlsKeyPair = key.tlsKeyPair(); | ||
if (mode == SslContextMode.SERVER) { | ||
assert tlsKeyPair != null; | ||
return createSslContext( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note: the call to createSslContext
modifies the global security providers during run-time (as opposed to start-up time when the ClientFactory
is initialized).
If users aren't using pure armeria, I think it's possible that MinifiedBouncyCastleProvider
can be used inadvertently.
I'm not sure if this is a good idea, but I don't see a way around this so SGTM for now
core/src/main/java/com/linecorp/armeria/common/TlsProvider.java
Outdated
Show resolved
Hide resolved
* Creates a new {@link TlsKeyPair} from the specified key file, key password and certificate chain | ||
* file. | ||
*/ | ||
public static TlsKeyPair of(File keyFile, @Nullable String keyPassword, File certificateChainFile) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note: while this is a convenient way to load a TlsKeyPair
, we may want to note in javadocs that TlsProvider#find
shouldn't block.
e.g.
Server.builder()
.tlsSelfSigned()
.tlsProvider(hostname -> TlsKeyPair.of(new File(""), new File("")))
.build();
return refCnt == 0; | ||
} | ||
|
||
void destroy() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Question) Is there no need for ReferenceCountUtil.release(sslContext)
in case SslContext
is an instance of ReferenceCountedOpenSslServerContext
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ReferenceCountedOpenSslServerContext
can't be used in Armeria. I added a TODO to support it.
https://github.com/line/armeria/pull/5228/files#diff-3fb600f6cb71c42141597949d05d9e1a8879ada1c3385f3aaaac9c774f5d0038R74-R75
@@ -420,6 +440,8 @@ public ClientFactoryBuilder tls(KeyManagerFactory keyManagerFactory) { | |||
@Override | |||
public ClientFactoryBuilder tlsCustomizer(Consumer<? super SslContextBuilder> tlsCustomizer) { | |||
requireNonNull(tlsCustomizer, "tlsCustomizer"); | |||
ensureNoTlsProvider(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Question) Is there a reason tlsCustomizer
isn't deprecated? It seems like all functionality is covered by ClientTlsConfig
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I imagined TlsProvider
is an optional feature for the TLS handshake.
For users who don't need dynamic TLS, ClientFactoryBuilder.tls(TlsKeyPair)
and ClientFactoryBuilder.tlsCustomizer()
would be enough for simplicity.
However, the API may be deprecated in the future.
* <p>Note that this method mutually exclusive with {@link #tls(TlsKeyPair)} and other static TLS settings. | ||
*/ | ||
@UnstableApi | ||
public ServerBuilder tlsProvider(TlsProvider tlsProvider) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note: I still think it may be useful for VirtualHost
s to set TlsProvider
since
- The hostname pattern can be set with a wildcard.
- Users may host multiple domains with a single server, and want to rotate each domain independently
I'm curious how hard it would be to modify s.t. each VirtualHost
can set a TlsProvider
:
- The defaultVirtualHost can set either a SslContext or a TlsProvider
- Each VirtualHost can set either a SslContext or a TlsProvider
- We define a
DelegatingTlsMapping
where eachVirtualHost.hostnamePattern
maps to aTlsProvider
or aSslContext
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is possible to support TlsProvider
for VirtualHost
s. I didn't implement it because a TlsProvider
can contain multiple certificates so one TlsProvider
covers all VirtualHost
s.
If there are any requirements for it, I'd be happy to implement it.
- Return associated certificates in `MappedTlsProvider` - Rename `TlsProvider.set{Default}()` to `TlsProvider.keyPair()`
…my4-dev-issue-5033
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍 👍 👍
@ikhoon is this ready for merging? |
Yes. |
Motivation:
#5033
API design note:
TlsKeyPair
represents a pair ofPrivateKey
andX509Certificate
chain.TlsSetters
have been deprecated in favor ofTlsSetters tls(TlsKeyPair)
.TlsProvider
dynamically resolves aTlsKeyPair
for the given hostname when a connection is established.*
is used as a special hostname to get theTlsKeyPair
for the default virtual host.TlsKeyPair
, a customTlsProvider
can be implemented.ServerTlsConfig
andClientTlsConfig
are added to override the default values and customizeSslContextBuilder
.TlsProvider
,*TlsConfig
are immutable so allTlsKeyPair
s returned by aTlsProvider
buildSslContext
with the same configuration.TlsProvider
andTlsKeyPair
for TLS configurations.SslContext
dynamically.Modifications:
TlsProviderMapping
that convertsTlsProvider
into SslContextMapping
forSniHandler
.TlsProvider
can be used to update the certificates withoutServer.reconfigure()
.TlsProvider
toServerBuilder
.VirtualHost
isn't added because aTlsProvider
can contain multiple certificates.TlsProvider
at the virtual host level later.Bootstraps
to create aBootstrap
with aTlsKeyPair
returned byTlsProvider
when a new connection is created.TlsProvider
is set, the original behavior that returns predefinedBootStraap
is used.TlsProvider
toClientFactoryBuilder
.TlsProvider
provides separate builders for the client and server.TlsKeyPair
provides various factory methods to easily create a key pair from different resources.SslContext
s and expire them after 1 hour of inactivity.CloseableMeterBinder
to unregister when the associated resource is unused.TlsSetters tls(File keyCertChainFile, File keyFile)
TlsSetters tls(File keyCertChainFile, File keyFile, @Nullable String keyPassword)
TlsSetters tls(InputStream keyCertChainInputStream, InputStream keyInputStream)
TlsSetters tls(InputStream keyCertChainInputStream, InputStream keyInputStream, @Nullable String keyPassword)
TlsSetters tls(PrivateKey key, X509Certificate... keyCertChain)
TlsSetters tls(PrivateKey key, Iterable<? extends X509Certificate> keyCertChain)
TlsSetters tls(PrivateKey key, @Nullable String keyPassword, X509Certificate... keyCertChain)
TlsSetters tls(PrivateKey key, @Nullable String keyPassword, Iterable<? extends X509Certificate> keyCertChain)
Result:
ClientFactory
#5033TlsProvider