Default logging format is not cleared properly

[stefangluszek]

Terraform Version

Run terraform -v to show the version. If you are not running the latest version of Terraform, please upgrade because your issue may have already been fixed.

Terraform v1.0.8

Affected Resource(s)

Please list the resources as a list, for example:

• fastly_service_v1

If this issue appears to affect multiple resources, it may be an issue with Terraform's core, so please mention this.

Terraform Configuration Files

resource "fastly_service_v1" "stefang_playground" {
logging_datadog {
    name               = "datadog"
    placement          = "none"
    format_version     = 2
    }
}

Debug Output

Panic Output

Expected Behavior

When no format is defined the default format should be cleared and there should be no diff on future runs of terraform apply.

Actual Behavior

terraform apply will detect a diff on every run caused by the default format field not being cleared.

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

1. Add login_datadog to your fastly_service_v1 without a format field

resource "fastly_service_v1" "stefang_playground" {
logging_datadog {
    name               = "datadog"
    placement          = "none"
    format_version     = 2
    }
}

2. Run terraform apply. The logging config will be added to your service

      + logging_datadog {
          + format_version = 2
          + name           = "datadog"
          + placement      = "none"
        }

3. Run terraform plan, since you've just applied your changes there should be no diff, but unfortunately, because the format still has the default value we will be presented with the following diff:

      + logging_datadog {
          + format_version = 2
          + name           = "datadog"
          + placement      = "none"
          + region         = "US"
        }
      - logging_datadog {
          - format         = <<-EOT
                {
                  "ddsource": "fastly",
                  "service": "%{req.service_id}V",
                  "date": "%{begin:%Y-%m-%dT%H:%M:%S%Z}t",
                  "time_start": "%{begin:%Y-%m-%dT%H:%M:%S%Z}t",
                  "time_end": "%{end:%Y-%m-%dT%H:%M:%S%Z}t",
                  "http": {
                    "request_time_ms": %D,
                    "method": "%m",
                    "url": "%{json.escape(req.url)}V",
                    "useragent": "%{User-Agent}i",
                    "referer": "%{Referer}i",
                    "protocol": "%H",
                    "request_x_forwarded_for": "%{X-Forwarded-For}i",
                    "status_code": "%s"
                  },
                  "network": {
                    "client": {
                     "ip": "%h",
                     "name": "%{client.as.name}V",
                     "number": "%{client.as.number}V",
                     "connection_speed": "%{client.geo.conn_speed}V"
                    },
                   "destination": {
                     "ip": "%A"
                    },
                  "geoip": {
                  "geo_city": "%{client.geo.city.utf8}V",
                  "geo_country_code": "%{client.geo.country_code}V",
                  "geo_continent_code": "%{client.geo.continent_code}V",
                  "geo_region": "%{client.geo.region}V"
                  },
                  "bytes_written": %B,
                  "bytes_read": %{req.body_bytes_read}V
                  },
                  "host": "%{Fastly-Orig-Host}i",
                  "origin_host": "%v",
                  "is_ipv6": %{if(req.is_ipv6, "true", "false")}V,
                  "is_tls": %{if(req.is_ssl, "true", "false")}V,
                  "tls_client_protocol": "%{json.escape(tls.client.protocol)}V",
                  "tls_client_servername": "%{json.escape(tls.client.servername)}V",
                  "tls_client_cipher": "%{json.escape(tls.client.cipher)}V",
                  "tls_client_cipher_sha": "%{json.escape(tls.client.ciphers_sha)}V",
                  "tls_client_tlsexts_sha": "%{json.escape(tls.client.tlsexts_sha)}V",
                  "is_h2": %{if(fastly_info.is_h2, "true", "false")}V,
                  "is_h2_push": %{if(fastly_info.h2.is_push, "true", "false")}V,
                  "h2_stream_id": "%{fastly_info.h2.stream_id}V",
                  "request_accept_content": "%{Accept}i",
                  "request_accept_language": "%{Accept-Language}i",
                  "request_accept_encoding": "%{Accept-Encoding}i",
                  "request_accept_charset": "%{Accept-Charset}i",
                  "request_connection": "%{Connection}i",
                  "request_dnt": "%{DNT}i",
                  "request_forwarded": "%{Forwarded}i",
                  "request_via": "%{Via}i",
                  "request_cache_control": "%{Cache-Control}i",
                  "request_x_requested_with": "%{X-Requested-With}i",
                  "request_x_att_device_id": "%{X-ATT-Device-Id}i",
                  "content_type": "%{Content-Type}o",
                  "is_cacheable": %{if(fastly_info.state~"^(HIT|MISS)$", "true","false")}V,
                  "response_age": "%{Age}o",
                  "response_cache_control": "%{Cache-Control}o",
                  "response_expires": "%{Expires}o",
                  "response_last_modified": "%{Last-Modified}o",
                  "response_tsv": "%{TSV}o",
                  "server_datacenter": "%{server.datacenter}V",
                  "req_header_size": %{req.header_bytes_read}V,
                  "resp_header_size": %{resp.header_bytes_written}V,
                  "socket_cwnd": %{client.socket.cwnd}V,
                  "socket_nexthop": "%{client.socket.nexthop}V",
                  "socket_tcpi_rcv_mss": %{client.socket.tcpi_rcv_mss}V,
                  "socket_tcpi_snd_mss": %{client.socket.tcpi_snd_mss}V,
                  "socket_tcpi_rtt": %{client.socket.tcpi_rtt}V,
                  "socket_tcpi_rttvar": %{client.socket.tcpi_rttvar}V,
                  "socket_tcpi_rcv_rtt": %{client.socket.tcpi_rcv_rtt}V,
                  "socket_tcpi_rcv_space": %{client.socket.tcpi_rcv_space}V,
                  "socket_tcpi_last_data_sent": %{client.socket.tcpi_last_data_sent}V,
                  "socket_tcpi_total_retrans": %{client.socket.tcpi_total_retrans}V,
                  "socket_tcpi_delta_retrans": %{client.socket.tcpi_delta_retrans}V,
                  "socket_ploss": %{client.socket.ploss}V
                }
            EOT -> null
          - format_version = 2 -> null
          - name           = "datadog" -> null
          - placement      = "none" -> null
          - region         = "US" -> null
        }

3. Setting format to an empty string does not help either

logging_datadog {
    name               = "datadog"
    placement          = "none"
    format_version     = 2
    format = ""
    }

4. I can work around the issue by using any non empty string for format

logging_datadog {
    name               = "datadog"
    placement          = "none"
    format_version     = 2
    format = "# PLEASE STORE THIS"
    }

5. After applying the above format change running terraform plan will show no diff \o/

I suspect this is actually not an issue with terraform provider but with the underlying Fastly API as you can reproduce the same behavior with Fastly web UI which won't allow you to clear the default format for logging.

[smaeda-ks]

You're correct that our API will inject default logging format if the format parameter is not set via POST request and hence this diff loop unless you explicitly specify your format in your TF file.

I patched a similar issue for other blocks before so I think we can do something similar for this as well. I'll look into this. Thanks

[pathikritmodak]

Is this being worked on?

[smaeda-ks]

Not yet. But I'll see if I can find some time to work on this in the coming weeks.

[philnielsen]

We are also hitting this perma diff :(