Reload fails when sending metrics via OTLP to itself

[Amoolaa]

Describe the bug
Reloading configuration by sending SIGHUP fails unexpectedly when sending internal metrics over OTLP (gRPC) to itself. You get the message:

Error: failed to shutdown the retiring config: failed to shutdown meter provider: failed to upload metrics: context deadline exceeded: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:4317: connect: connection refused

Steps to reproduce
Send SIGHUP to otelcol process exporting metrics to itself.

What did you expect to see?
Otelcol to gracefully reload

What did you see instead?

{"level":"info","ts":1730698222.0050309,"caller":"[email protected]/collector.go:328","msg":"Received signal from OS","signal":"hangup"}
{"level":"warn","ts":1730698222.0075464,"caller":"[email protected]/collector.go:237","msg":"Config updated, restart service"}
{"level":"info","ts":1730698222.007997,"caller":"[email protected]/service.go:266","msg":"Starting shutdown..."}
{"level":"info","ts":1730698222.0132873,"caller":"healthcheck/handler.go:132","msg":"Health Check state change","kind":"extension","name":"health_check","status":"unavailable"}
{"level":"info","ts":1730698222.018832,"caller":"adapter/receiver.go:126","msg":"Stopping stanza receiver","kind":"receiver","name":"filelog/system","data_type":"logs"}
{"level":"info","ts":1730698222.019587,"caller":"extensions/extensions.go:66","msg":"Stopping extensions..."}
{"level":"info","ts":1730698222.01967,"caller":"[email protected]/service.go:280","msg":"Shutdown complete."}
{"level":"warn","ts":1730698222.0267143,"caller":"[email protected]/clientconn.go:1379","msg":"[core] [Channel #1 SubChannel #4]grpc: addrConn.createTransport failed to connect to {Addr: \"127.0.0.1:4317\", ServerName: \"localhost:4317\", }. Err: connection error: desc = \"transport: Error while dialing: dial tcp 127.0.0.1:4317: connect: connection refused\"","grpc_log":true}
{"level":"warn","ts":1730698223.028053,"caller":"[email protected]/clientconn.go:1379","msg":"[core] [Channel #1 SubChannel #4]grpc: addrConn.createTransport failed to connect to {Addr: \"127.0.0.1:4317\", ServerName: \"localhost:4317\", }. Err: connection error: desc = \"transport: Error while dialing: dial tcp 127.0.0.1:4317: connect: connection refused\"","grpc_log":true}
{"level":"warn","ts":1730698224.8170724,"caller":"[email protected]/clientconn.go:1379","msg":"[core] [Channel #1 SubChannel #4]grpc: addrConn.createTransport failed to connect to {Addr: \"127.0.0.1:4317\", ServerName: \"localhost:4317\", }. Err: connection error: desc = \"transport: Error while dialing: dial tcp 127.0.0.1:4317: connect: connection refused\"","grpc_log":true}
{"level":"warn","ts":1730698227.1291044,"caller":"[email protected]/clientconn.go:1379","msg":"[core] [Channel #1 SubChannel #4]grpc: addrConn.createTransport failed to connect to {Addr: \"127.0.0.1:4317\", ServerName: \"localhost:4317\", }. Err: connection error: desc = \"transport: Error while dialing: dial tcp 127.0.0.1:4317: connect: connection refused\"","grpc_log":true}
{"level":"warn","ts":1730698231.443441,"caller":"[email protected]/clientconn.go:1379","msg":"[core] [Channel #1 SubChannel #4]grpc: addrConn.createTransport failed to connect to {Addr: \"127.0.0.1:4317\", ServerName: \"localhost:4317\", }. Err: connection error: desc = \"transport: Error while dialing: dial tcp 127.0.0.1:4317: connect: connection refused\"","grpc_log":true}

Error: failed to shutdown the retiring config: failed to shutdown meter provider: failed to upload metrics: context deadline exceeded: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:4317: connect: connection refused

What version did you use?
v0.112.0 contrib distribution

What config did you use?

receivers:
  otlp:
    protocols:
      http:
        endpoint: 0.0.0.0:4318
      grpc:
        endpoint: 0.0.0.0:4317
...
service:
  extensions:
    - health_check
  pipelines:
    metrics:
      receivers:
        - otlp
      exporters:
        - prometheusremotewrite/lgtm
service:
  telemetry:
    metrics:
      level: detailed
      readers:
        - periodic:
            interval: 60000
            exporter:
              otlp:
                protocol: grpc/protobuf
                endpoint: localhost:4317