Provide an option to verify propagation against a custom list of DNS-servers

[Jonher937]

Welcome

• Yes, I've searched similar issues on GitHub and didn't find any.

How do you use lego?

Binary and Traefik.

Detailed Description

Hi, we've hit an issue using DNS-01 challenge with EAB and CNAME delegation with EJBCA as the CA, but this deployment scenario is probably going to be more common as ACME has started to be adopted in enterprise.

I have included a diagram to try and explain our interpretation of the current flow, and where we see an issue.

1. We issue a renewal request, all good so far
2. We create the TXT record with the challenge
3. LEGO uses our --dns.resolvers that points to the same DNS-server as the PKI will query for the challenge. These servers are used by lego to find the authoritative servers only as specified here.
   It finds the authoritative servers and can successfully query to verify they serve the TXT record with the correct value.
4. LEGO gives the all clear to the PKI
5. PKI queries DNS against the company wide DNS-service. This company DNS-service has network communication to the authoritative DNS-servers over tcp/udp 53. Queries are cached and if forwarded.

So now to the problem:

• Step 5 we can see the PKI is NOT querying the authoritative DNS-server. This is by network security design.
• In Step 3 we would like to validate the same record as we added to the authoritative DNS-server has propagated to the DNS-servers used by the PKI before giving the "All good" message to the PKI.

Essentially we'd want something like this (behind a flag such as --dns.propagation-servers of []string type) to specify what additional servers we'd want to append to the checkAuthoritativeNss function.

https://github.com/Jonher937/lego/blob/a152249a1a02146604936099d3bf1a9d13999280/challenge/dns01/precheck.go#L76-L88

Full commit can be found here and acts as an example. I have not found a good way to pass down a flag this deep into the process, but this issue is created as a topic for discussing how and if this could be implemented.

In the end our issues comes down to the DNS topology/implementation and slow (60+ seconds) propagations to the company DNS-service which the PKI verifies against.

We have also tries with the newly added --dns.propagation-wait option and have successfully managed to obtain certificates if we tweak it high enough for the propagation to happen, but this might randomly fail if the zone update has not yet made it's way to the company DNS-service.

[ldez]

Hello,

you can already do that:

• --dns.resolvers allows defining the DNS used to check the propagation
• --dns.disable-cp allows to skip the "classic" authoritative NSs.

[Jonher937]

We have tried the disable-cp option: By setting this flag to true, disables the need to await propagation of the TXT record to all authoritative name servers.

Our problem is the reverse, the authoritative name servers replies correct and do so straight away. But the servers the PKI uses for validation are slow to serve the record.

What we need to do is either:

• Sleep for long enough before blindly telling the PKI the record has "propagated" using the dns.propagation-wait option and hope that it has propagated to the DNS-service used by the PKI to do the TXT lookup
• OR we need to verify using lego against the DNS-service's servers that the record has propagated rather than depending on a time.Duration which does not guarantee it's there

--dns.resolvers option documents it's only used to:

Set the resolvers to use for performing (recursive) CNAME resolving and apex domain determination.
For DNS-01 challenge verification, the authoritative DNS server is queried directly.
source for the quote

I tried using the options you suggested but it does not wait for any propagation at all, unless of course I use --dns.propagation-wait in addition to those values.
We need to verify the propagation has occured on the non-authoritative DNS-service before telling the PKI it's ready to do the DNS validation. If we don't the PKI will check with DNS-service which does not know about this TXT value and it will fail the issuing of the certificate.

[Jonher937]

Today we looked into why cert-manager has had more success and it looks like cert-manager has this option: dns01-recursive-nameservers-only documented here

--dns01-recursive-nameservers-only Forces cert-manager to only use the recursive nameservers for verification. Enabling this option could cause the DNS01 self check to take longer due to caching performed by the recursive nameservers.

• https://github.com/cert-manager/cert-manager/blob/6d2f5e145ec54c0cb531cedabcfdcd1aad76fea3/cmd/controller/app/controller.go#L332
• https://github.com/cert-manager/cert-manager/blob/master/pkg/issuer/acme/dns/dns.go#L120-L121 here s.Context.DNS01CheckAuthoritative would be false, which means util.PreCheckDNS will use the user-specified nameservers instead of the Authoritative, just like we would want with LEGO.
  • https://github.com/cert-manager/cert-manager/blob/master/pkg/issuer/acme/dns/util/wait.go#L107-L109