class Google::Cloud::Env::ComputeMetadata

A client for the Google metadata service.

Constants

DEFAULT_HOST

The default host for the metadata server @return [String]

DEFAULT_OPEN_TIMEOUT

The default timeout in seconds for opening http connections @return [Numeric]

DEFAULT_REQUEST_TIMEOUT

The default timeout in seconds for request responses @return [Numeric]

DEFAULT_RETRY_COUNT

The default number of retries @return [Integer]

DEFAULT_RETRY_INTERVAL

The default interval between retries, in seconds @return [Numeric]

DEFAULT_RETRY_TIMEOUT

The default timeout across retries @return [nil]

DEFAULT_WARMUP_TIME

The default time in seconds to wait for environment warmup. @return [Numeric]

FLAVOR_HEADER

@private The standard set of headers @return [Hash{String=>String}]

PATH_BASE

@private The base path of metadata server queries. @return [String]

TOKEN_EXPIRY_BUFFER

@private

A buffer in seconds for token expiry. Our cache for the token will expire approximately this many seconds before the declared expiry time of the token itself.

We want this value to be positive so that we provide some buffer to offset any clock skew and Metadata Server latency that might affect our calculation of the expiry time, but more importantly so that a client has approximately this amount of time to use a token we give them before it expires.

We don’t want this to be much higher, however, to keep the load down on the Metadata Server. We’ve been advised by the compute/serverless engineering teams to set this value less than 4 minutes because the Metadata Server can refresh the token as late as 4 minutes before the actual expiry of the previous token. If our cache expires and we request a new token, we actually want to receive a new token rather than the previous old token. See internal issue b/311414224.

TRANSIENT_EXCEPTIONS

@private A list of exceptions that are considered transient. They trigger a retry if received from an HTTP attempt, and they are not cached (i.e. the cache lifetime is set to 0.)

Attributes

cache[R]

@private The underlying LazyDict. Can be used to customize the cache for testing. @return [Google::Cloud::Env::LazyDict]

compute_smbios[R]

@private The compute SMBIOS access object @return [Google::Cloud::Env::ComputeSMBIOS]

connection[R]

@private The underlying Faraday connection. Can be used to customize the connection for testing. @return [Faraday::Connection]

host[R]

The host URL for the metadata server, including ‘http://`.

@return [String]

overrides[R]

The overrides, or nil if overrides are not present. If present, overrides will answer all metadata queries, and actual calls to the metadata server will be blocked.

@return [Overrides,nil]

retry_count[RW]

The default maximum number of times to retry a query for a key. A value of 1 means 2 attempts (i.e. 1 retry). A value of nil means there is no limit to the number of retries, although there could be an overall timeout.

Defaults to {DEFAULT_RETRY_COUNT}.

@return [Integer,nil]

retry_interval[RW]

The time in seconds between retries. This time includes the time spent by the previous attempt.

Defaults to {DEFAULT_RETRY_INTERVAL}.

@return [Numeric]

retry_timeout[RW]

The default overall timeout across all retries of a lookup, in seconds. A value of nil means there is no timeout, although there could be a limit to the number of retries.

Defaults to {DEFAULT_RETRY_TIMEOUT}.

@return [Numeric,nil]

variables[R]

@private The variables access object @return [Google::Cloud::Env::Variables]

warmup_time[RW]

A time in seconds allotted to environment warmup, during which retries will not be ended. This handles certain environments in which the Metadata Server might not be fully awake until some time after application startup. A value of nil disables this warmup period.

Defaults to {DEFAULT_WARMUP_TIME}.

@return [Numeric,nil]

Public Class Methods

new(variables: nil, compute_smbios: nil) click to toggle source

Create a compute metadata access object.

@param variables [Google::Cloud::Env::Variables] Access object for

environment variables. If not provided, a default is created.

@param compute_smbios [Google::Cloud::Env::ComputeSMBIOS] Access

object for SMBIOS information. If not provided, a default is
created.
# File lib/google/cloud/env/compute_metadata.rb, line 247
def initialize variables: nil,
               compute_smbios: nil
  @variables = variables || Variables.new
  @compute_smbios = compute_smbios || ComputeSMBIOS.new
  # This mutex protects the overrides and existence settings.
  # Those values won't change within a synchronize block.
  @mutex = Thread::Mutex.new
  reset!
end

Public Instance Methods

check_existence(open_timeout: nil, request_timeout: nil, retry_count: :default, retry_timeout: :default) click to toggle source

Return detailed information about whether we think Metadata is available. If we have not previously confirmed existence one way or another, this could block while trying to contact the server through the given timeouts and retries.

@param open_timeout [Numeric] Timeout for opening http connections.

Defaults to {#open_timeout}.

@param request_timeout [Numeric] Timeout for entire http requests.

Defaults to {#request_timeout}.

@param retry_count [Integer,nil] Number of times to retry. A value of

1 means 2 attempts (i.e. 1 retry). A value of nil indicates
retries are limited only by the timeout. Defaults to
{#retry_count}.

@param retry_timeout [Numeric,nil] Total timeout for retries. A value

of nil indicates no time limit, and retries are limited only by
count. Defaults to {#retry_timeout}.

@return [:no] if we know the metadata server is not present @return [:unconfirmed] if we believe metadata should be present but we

haven't gotten a confirmed response from it. This can happen if
SMBIOS says we're on GCE but we can't contact the Metadata Server
even through retries.

@return [:confirmed] if we have a confirmed response from metadata.

# File lib/google/cloud/env/compute_metadata.rb, line 480
def check_existence open_timeout: nil,
                    request_timeout: nil,
                    retry_count: :default,
                    retry_timeout: :default
  current = @existence
  return current if [:no, :confirmed].include? @existence
  begin
    lookup nil,
           open_timeout: open_timeout,
           request_timeout: request_timeout,
           retry_count: retry_count,
           retry_timeout: retry_timeout
  rescue MetadataServerNotResponding
    # Do nothing
  end
  @existence
end
ensure_existence(timeout: nil) click to toggle source

Assert that the Metadata Server should be present, and wait for a confirmed connection to ensure it is up. This will generally run at most {#warmup_time} seconds to wait out the expected maximum warmup time, but a shorter timeout can be provided.

@param timeout [Numeric,nil] a timeout in seconds, or nil to wait

until we have conclusively decided one way or the other.

@return [:confirmed] if we were able to confirm connection. @raise [MetadataServerNotResponding] if we were unable to confirm

connection with the Metadata Server, either because the timeout
expired or because the server seems to be down
# File lib/google/cloud/env/compute_metadata.rb, line 525
def ensure_existence timeout: nil
  timeout ||= @startup_time + warmup_time - Process.clock_gettime(Process::CLOCK_MONOTONIC)
  timeout = 1.0 if timeout < 1.0
  check_existence retry_count: nil, retry_timeout: timeout
  raise MetadataServerNotResponding unless @existence == :confirmed
  @existence
end
existence_immediate() click to toggle source

The current detailed existence status, without blocking on any attempt to contact the metadata server.

@return [nil] if we have no information at all yet @return [:no] if we know the metadata server is not present @return [:unconfirmed] if we believe metadata should be present but we

haven't gotten a confirmed response from it.

@return [:confirmed] if we have a confirmed response from metadata.

# File lib/google/cloud/env/compute_metadata.rb, line 508
def existence_immediate
  @existence
end
expiration_time_of(path, query: nil) click to toggle source

Get the expiration time for the given path. Returns the monotonic time if the data has been retrieved and has an expiration, nil if the data has been retrieved but has no expiration, or false if the data has not yet been retrieved.

@return [Numeric,nil,false]

# File lib/google/cloud/env/compute_metadata.rb, line 541
def expiration_time_of path, query: nil
  state = @cache.internal_state [path, query]
  return false unless state[0] == :success
  state[2]
end
host=(new_host) click to toggle source

The host URL for the metadata server, including ‘http://`.

@param new_host [String]

# File lib/google/cloud/env/compute_metadata.rb, line 269
def host= new_host
  new_host ||= @variables["GCE_METADATA_HOST"] || DEFAULT_HOST
  new_host = "http://#{new_host}" unless new_host.start_with? "http://"
  @host = new_host
end
lookup(path, query: nil, open_timeout: nil, request_timeout: nil, retry_count: :default, retry_timeout: :default) click to toggle source

Look up a particular key from the metadata server and return the data as a string. Could return a cached value if the key has been queried before, otherwise this could block while trying to contact the server through the given timeouts and retries.

This returns the HTTP body as a string, only if the call succeeds. If the key is inaccessible or missing (i.e. the HTTP status was not 200) or does not have the correct ‘Metadata-Flavor` header, then nil is returned. If you need more detailed information, use {#lookup_response}.

@param path [String] The key path (e.g. ‘project/project-id`) @param query [Hash{String => String}] Any additional query parameters

to send with the request.

@param open_timeout [Numeric] Timeout for opening http connections.

Defaults to {#open_timeout}.

@param request_timeout [Numeric] Timeout for entire http requests.

Defaults to {#request_timeout}.

@param retry_count [Integer,nil] Number of times to retry. A value of

1 means 2 attempts (i.e. 1 retry). A value of nil indicates
retries are limited only by the timeout. Defaults to
{#retry_count}.

@param retry_timeout [Numeric,nil] Total timeout for retries. A value

of nil indicates no time limit, and retries are limited only by
count. Defaults to {#retry_timeout}.

@return [String] the data from the metadata server @return [nil] if the key is not present @raise [MetadataServerNotResponding] if the Metadata Server is not

responding
# File lib/google/cloud/env/compute_metadata.rb, line 439
def lookup path,
           query: nil,
           open_timeout: nil,
           request_timeout: nil,
           retry_count: :default,
           retry_timeout: :default
  response = lookup_response path,
                             query: query,
                             open_timeout: open_timeout,
                             request_timeout: request_timeout,
                             retry_count: retry_count,
                             retry_timeout: retry_timeout
  return nil unless response.status == 200 && response.google_flavor?
  response.body
end
lookup_response(path, query: nil, open_timeout: nil, request_timeout: nil, retry_count: :default, retry_timeout: :default) click to toggle source

Look up a particular key from the metadata server, and return a full {Response} object. Could return a cached value if the key has been queried before, otherwise this could block while trying to contact the server through the given timeouts and retries.

This returns a Response object even if the HTTP status is 404, so be sure to check the status code to determine whether the key actually exists. Unlike {#lookup}, this method does not return nil.

@param path [String] The key path (e.g. ‘project/project-id`) @param query [Hash{String => String}] Any additional query parameters

to send with the request.

@param open_timeout [Numeric] Timeout for opening http connections.

Defaults to {#open_timeout}.

@param request_timeout [Numeric] Timeout for entire http requests.

Defaults to {#request_timeout}.

@param retry_count [Integer,nil] Number of times to retry. A value of

1 means 2 attempts (i.e. 1 retry). A value of nil indicates
retries are limited only by the timeout. Defaults to
{#retry_count}.

@param retry_timeout [Numeric,nil] Total timeout for retries. A value

of nil indicates no time limit, and retries are limited only by
count. Defaults to {#retry_timeout}.

@return [Response] the data from the metadata server @raise [MetadataServerNotResponding] if the Metadata Server is not

responding
# File lib/google/cloud/env/compute_metadata.rb, line 385
def lookup_response path,
                    query: nil,
                    open_timeout: nil,
                    request_timeout: nil,
                    retry_count: :default,
                    retry_timeout: :default
  query = canonicalize_query query
  if @overrides
    @mutex.synchronize do
      return lookup_override path, query if @overrides
    end
  end
  raise MetadataServerNotResponding unless gce_check
  retry_count = self.retry_count if retry_count == :default
  retry_count += 1 if retry_count
  retry_timeout = self.retry_timeout if retry_timeout == :default
  @cache.await [path, query], open_timeout, request_timeout,
               transient_errors: [MetadataServerNotResponding],
               max_tries: retry_count,
               max_time: retry_timeout
end
open_timeout() click to toggle source

The timeout for opening http connections in seconds.

@return [Numeric]

# File lib/google/cloud/env/compute_metadata.rb, line 325
def open_timeout
  connection.options.open_timeout
end
open_timeout=(timeout) click to toggle source

The timeout for opening http connections in seconds.

@param timeout [Numeric]

# File lib/google/cloud/env/compute_metadata.rb, line 334
def open_timeout= timeout
  connection.options[:open_timeout] = timeout
end
overrides=(new_overrides) click to toggle source

Set the overrides. You can also set nil to disable overrides. If present, overrides will answer all metadata queries, and actual calls to the metadata server will be blocked.

@param new_overrides [Overrides,nil]

# File lib/google/cloud/env/compute_metadata.rb, line 563
def overrides= new_overrides
  @mutex.synchronize do
    @existence = nil
    @overrides = new_overrides
  end
end
request_timeout() click to toggle source

The total timeout for an HTTP request in seconds.

@return [Numeric]

# File lib/google/cloud/env/compute_metadata.rb, line 343
def request_timeout
  connection.options.timeout
end
request_timeout=(timeout) click to toggle source

The total timeout for an HTTP request in seconds.

@param timeout [Numeric]

# File lib/google/cloud/env/compute_metadata.rb, line 352
def request_timeout= timeout
  connection.options[:timeout] = timeout
end
reset!() click to toggle source

@private Reset the cache, overrides, and all settings to default, for testing.

# File lib/google/cloud/env/compute_metadata.rb, line 630
def reset!
  @mutex.synchronize do
    self.host = nil
    @connection = Faraday.new url: host
    self.open_timeout = DEFAULT_OPEN_TIMEOUT
    self.request_timeout = DEFAULT_REQUEST_TIMEOUT
    self.retry_count = DEFAULT_RETRY_COUNT
    self.retry_timeout = DEFAULT_RETRY_TIMEOUT
    self.retry_interval = DEFAULT_RETRY_INTERVAL
    self.warmup_time = DEFAULT_WARMUP_TIME
    @cache = create_cache
    @overrides = nil
  end
  reset_existence!
end
reset_existence!() click to toggle source

@private Clear the existence cache, for testing.

# File lib/google/cloud/env/compute_metadata.rb, line 650
def reset_existence!
  @mutex.synchronize do
    @existence = nil
    @startup_time = Process.clock_gettime Process::CLOCK_MONOTONIC
  end
  self
end
with_overrides(temp_overrides) { || ... } click to toggle source

Run the given block with the overrides replaced with the given set (or nil to disable overrides in the block). The original overrides setting is restored at the end of the block. This is used for debugging/testing/mocking.

@param temp_overrides [Overrides,nil]

# File lib/google/cloud/env/compute_metadata.rb, line 578
def with_overrides temp_overrides
  old_overrides, old_existence = @mutex.synchronize do
    [@overrides, @existence]
  end
  begin
    @mutex.synchronize do
      @existence = nil
      @overrides = temp_overrides
    end
    yield
  ensure
    @mutex.synchronize do
      @existence = old_existence
      @overrides = old_overrides
    end
  end
end

Private Instance Methods

access_token_lifetime(data) click to toggle source

@private Extract the lifetime of an access token

# File lib/google/cloud/env/compute_metadata.rb, line 807
def access_token_lifetime data
  json = JSON.parse data rescue nil
  return 0 unless json.respond_to?(:key?) && json.key?("expires_in")
  lifetime = json["expires_in"].to_i - TOKEN_EXPIRY_BUFFER
  lifetime = 0 if lifetime.negative?
  lifetime
end
canonicalize_query(query) click to toggle source

@private Stringify keys in a query hash

# File lib/google/cloud/env/compute_metadata.rb, line 833
def canonicalize_query query
  query&.transform_keys(&:to_s)
end
create_cache() click to toggle source

@private Create and return a new LazyDict cache for the metadata

# File lib/google/cloud/env/compute_metadata.rb, line 737
def create_cache
  retries = proc do
    Google::Cloud::Env::Retries.new max_tries: nil,
                                    initial_delay: retry_interval,
                                    delay_includes_time_elapsed: true
  end
  Google::Cloud::Env::LazyDict.new retries: retries do |(path, query), open_timeout, request_timeout|
    internal_lookup path, query, open_timeout, request_timeout
  end
end
determine_data_lifetime(path, data) click to toggle source

@private Compute the lifetime of data, given the path and data. Returns the value in seconds, or nil for nonexpiring data.

# File lib/google/cloud/env/compute_metadata.rb, line 794
def determine_data_lifetime path, data
  case path
  when %r{instance/service-accounts/[^/]+/token}
    access_token_lifetime data
  when %r{instance/service-accounts/[^/]+/identity}
    identity_token_lifetime data
  end
end
gce_check() click to toggle source

@private

Attempt to determine if we’re on GCE (if we haven’t previously), and update the existence flag. Return true if we could be on GCE, or false if we’re definitely not.

# File lib/google/cloud/env/compute_metadata.rb, line 704
def gce_check
  if @existence.nil?
    @mutex.synchronize do
      @existence ||=
        if @compute_smbios.google_compute? || maybe_gcf || maybe_gcr || maybe_gae
          :unconfirmed
        else
          :no
        end
    end
  end
  @existence != :no
end
identity_token_lifetime(data) click to toggle source

@private Extract the lifetime of an identity token

# File lib/google/cloud/env/compute_metadata.rb, line 819
def identity_token_lifetime data
  return 0 unless data =~ /^[\w=-]+\.([\w=-]+)\.[\w=-]+$/
  base64 = Base64.urlsafe_decode64 Regexp.last_match[1]
  json = JSON.parse base64 rescue nil
  return 0 unless json.respond_to?(:key?) && json&.key?("exp")
  lifetime = json["exp"].to_i - Time.now.to_i - TOKEN_EXPIRY_BUFFER
  lifetime = 0 if lifetime.negative?
  lifetime
end
internal_lookup(path, query, open_timeout, request_timeout) click to toggle source

@private Look up the given path, without using the cache.

# File lib/google/cloud/env/compute_metadata.rb, line 752
def internal_lookup path, query, open_timeout, request_timeout
  full_path = path ? "#{PATH_BASE}/#{path}" : ""
  http_response = connection.get full_path do |req|
    req.params = query if query
    req.headers = FLAVOR_HEADER
    req.options.timeout = request_timeout if request_timeout
    req.options.open_timeout = open_timeout if open_timeout
  end
  response = Response.new http_response.status, http_response.body, http_response.headers
  if path.nil?
    post_update_existence(response.status == 200 && response.google_flavor?, response.retrieval_monotonic_time)
  elsif response.google_flavor?
    post_update_existence true, response.retrieval_monotonic_time
  end
  lifetime = determine_data_lifetime path, response.body.strip
  LazyValue.expiring_value lifetime, response
rescue *TRANSIENT_EXCEPTIONS
  post_update_existence false
  raise MetadataServerNotResponding
end
lookup_override(path, query) click to toggle source

@private Lookup from overrides and return the result or raise. This must be called from within the mutex, and assumes that overrides is non-nil.

# File lib/google/cloud/env/compute_metadata.rb, line 843
def lookup_override path, query
  if @overrides.empty?
    @existence = :no
    raise MetadataServerNotResponding
  end
  @existence = :confirmed
  result = @overrides.lookup path, query: query
  result ||= Response.new 404, "Not found", FLAVOR_HEADER
  result
end
maybe_gae() click to toggle source

@private

# File lib/google/cloud/env/compute_metadata.rb, line 729
def maybe_gae
  @variables["GAE_SERVICE"] && @variables["GAE_RUNTIME"]
end
maybe_gcf() click to toggle source

@private

# File lib/google/cloud/env/compute_metadata.rb, line 719
def maybe_gcf
  @variables["K_SERVICE"] && @variables["K_REVISION"] && @variables["GAE_RUNTIME"]
end
maybe_gcr() click to toggle source

@private

# File lib/google/cloud/env/compute_metadata.rb, line 724
def maybe_gcr
  @variables["K_SERVICE"] && @variables["K_REVISION"] && @variables["K_CONFIGURATION"]
end
post_update_existence(success, current_time = nil) click to toggle source

@private Update existence based on a received result

# File lib/google/cloud/env/compute_metadata.rb, line 777
def post_update_existence success, current_time = nil
  return if @existence == :confirmed
  @mutex.synchronize do
    if success
      @existence = :confirmed
    elsif @existence != :confirmed
      current_time ||= Process.clock_gettime Process::CLOCK_MONOTONIC
      @existence = :no if current_time > @startup_time + warmup_time
    end
  end
end