SyntaxError: Could not find export 'fromNodeProviderChain' in module '@aws-sdk/credential-providers'

[paul-uz]

Despite this package beig included, I get this error when trying to run my code. Installing the package locally as a dev dependency, I get no errors about fromNodeProviderChain being missing, and can indeed see the export.

[paul-uz]

Ideally, could we get https://www.npmjs.com/package/@aws-sdk/credential-provider-node added to the runtime?

[nabetti1720]

I tried to add @aws-sdk/credential-provider-node but at the end of the dependency I got an error that the http module could not be resolved.

Optimized: CognitoIdentity
✘ [ERROR] Could not resolve "http"

    node_modules/@smithy/credential-provider-imds/dist-es/remoteProvider/httpRequest.js:3:24:
      3 │ import { request } from "http";
        ╵                         ~~~~~~

  The package "http" wasn't found on the file system but is built into node. Are you trying to
  bundle for node? You can use "platform: 'node'" to do that, which will remove this error.

Optimized: Aws_json1_1
Optimized: Aws_json1_1
Optimized: Aws_json1_1
Optimized: Aws_json1_1
Optimized: Aws_json1_1
Optimized: Aws_json1_1
Optimized: Aws_json1_1
Optimized: Aws_json1_1
1 error
/Users/shinya/Workspaces/llrt/node_modules/esbuild/lib/main.js:1472
  let error = new Error(text);
              ^

Error: Build failed with 1 error:
node_modules/@smithy/credential-provider-imds/dist-es/remoteProvider/httpRequest.js:3:24: ERROR: Could not resolve "http"
    at failureErrorWithLog (/Users/shinya/Workspaces/llrt/node_modules/esbuild/lib/main.js:1472:15)
    at /Users/shinya/Workspaces/llrt/node_modules/esbuild/lib/main.js:945:25
    at runOnEndCallbacks (/Users/shinya/Workspaces/llrt/node_modules/esbuild/lib/main.js:1315:45)
    at buildResponseToResult (/Users/shinya/Workspaces/llrt/node_modules/esbuild/lib/main.js:943:7)
    at /Users/shinya/Workspaces/llrt/node_modules/esbuild/lib/main.js:970:16
    at responseCallbacks.<computed> (/Users/shinya/Workspaces/llrt/node_modules/esbuild/lib/main.js:622:9)
    at handleIncomingPacket (/Users/shinya/Workspaces/llrt/node_modules/esbuild/lib/main.js:677:12)
    at Socket.readFromStdout (/Users/shinya/Workspaces/llrt/node_modules/esbuild/lib/main.js:600:7)
    at Socket.emit (node:events:519:28)
    at addChunk (node:internal/streams/readable:559:12) {
  errors: [Getter/Setter],
  warnings: [Getter/Setter]
}

Node.js v20.13.1

Perhaps this is why @aws-sdk/credential-provider-node cannot be added.

Currently, when building LLRT, the platform is browser. Would it make any difference if I set the platform of your application's bundler to browser?

[nabetti1720]

Related to #40

[paul-uz]

Ha it is because it should be marked as external or whatever terminology your bundler uses so it doesn't try to resolve it at bundling time. It is provided at runtime.

This isn't a helpful comment.

[richarddavison]

@paul-uz thanks for your comments. fromNodeProviderChain as the name implies assumes a Node.js environment. We could provide it but we're not supporting the required dependencies. What features are you looking for from fromNodeProviderChain? You should be able to use alternatives such as credential-provider-http, credential-provider-env, credential-provider-process etc

[paul-uz]

@paul-uz thanks for your comments. fromNodeProviderChain as the name implies assumes a Node.js environment. We could provide it but we're not supporting the required dependencies. What features are you looking for from fromNodeProviderChain? You should be able to use alternatives such as credential-provider-http, credential-provider-env, credential-provider-process etc

I ideally need the default provider from the other package, for signing requests.

Tbh I don't actually know which one I could use in its place as the one I was using tries various methods of finding the credentials and I don't know what lambda actually uses for node projects.

[richarddavison]

@paul-uz can you share a bit of code? I need some more context to understand what you're after. Usually you don't have to deal with the credential process in the SDK.

[paul-uz]

@paul-uz can you share a bit of code? I need some more context to understand what you're after. Usually you don't have to deal with the credential process in the SDK.

Sure here you go

      const request = new HttpRequest({
        headers: {
          'Content-Type': 'application/json',
          'host': SEARCH_DOMAIN_ENDPOINT!,
        },
        hostname: SEARCH_DOMAIN_ENDPOINT!,
        method: 'GET',
        path: SEARCH_INDEX + '/_search',
        query: queryParams as any,
      });

      var signer = new SignatureV4({
        credentials: defaultProvider(),
        region: REGION,
        service: 'es',
        sha256: Sha256,
      });

      const signedRequest = await signer.sign(request);
      const { response } = await client.handle(signedRequest as HttpRequest);

[paul-uz]

@richarddavison any updates on this? It would be nice to get our app using the LLRT

[richarddavison]

@richarddavison any updates on this? It would be nice to get our app using the LLRT

I will take a look at your example. I'm rather confident this would work with minimum modification!

[richarddavison]

@richarddavison any updates on this? It would be nice to get our app using the LLRT

Here is a local working example:

import { HttpRequest } from "@smithy/protocol-http";
import { SignatureV4 } from "@smithy/signature-v4";
import { Sha256 } from "@aws-crypto/sha256-browser";

const CREDENTIALS = {
  accessKeyId: process.env.AWS_ACCESS_KEY_ID,
  secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
  sessionToken: process.env.AWS_SESSION_TOKEN,
};

const REGION = process.env.AWS_REGION;

const SEARCH_DOMAIN_ENDPOINT = "https://www.example.com";
const SEARCH_INDEX = "index_name";
const queryParams = {
  example: "foobar",
};

const request = new HttpRequest({
  headers: {
    "Content-Type": "application/json",
    host: SEARCH_DOMAIN_ENDPOINT,
  },
  hostname: SEARCH_DOMAIN_ENDPOINT,
  method: "GET",
  path: SEARCH_INDEX + "/_search",
  query: queryParams,
});

var signer = new SignatureV4({
  credentials: CREDENTIALS,
  region: REGION,
  service: "es",
  sha256: Sha256,
});

const signedRequest = await signer.sign(request);

console.log(signedRequest);

[paul-uz]

@richarddavison any updates on this? It would be nice to get our app using the LLRT

Here is a local working example:

import { HttpRequest } from "@smithy/protocol-http";
import { SignatureV4 } from "@smithy/signature-v4";
import { Sha256 } from "@aws-crypto/sha256-browser";

const CREDENTIALS = {
  accessKeyId: process.env.AWS_ACCESS_KEY_ID,
  secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
  sessionToken: process.env.AWS_SESSION_TOKEN,
};

const REGION = process.env.AWS_REGION;

const SEARCH_DOMAIN_ENDPOINT = "https://www.example.com";
const SEARCH_INDEX = "index_name";
const queryParams = {
  example: "foobar",
};

const request = new HttpRequest({
  headers: {
    "Content-Type": "application/json",
    host: SEARCH_DOMAIN_ENDPOINT,
  },
  hostname: SEARCH_DOMAIN_ENDPOINT,
  method: "GET",
  path: SEARCH_INDEX + "/_search",
  query: queryParams,
});

var signer = new SignatureV4({
  credentials: CREDENTIALS,
  region: REGION,
  service: "es",
  sha256: Sha256,
});

const signedRequest = await signer.sign(request);

console.log(signedRequest);

Thank you for taking the time to reply, however, this code doesn't use the defaultProvider method which is what I need to use.

[richarddavison]

No problem. I might be missing something here but why do you need the defaultProvider function?

[paul-uz]

No problem. I might be missing something here but why do you need the defaultProvider function?

Simply the fact that that method is what we have historically used to provide credentials inside our Lambdas when needed. Ideally, the less refactoring required, the more likely that we can start using LLRT for certain/all functions.

If there is no feasible way it can be used currently with the LLRT due to missing packages ie http/https, then we'd need an easy solution that takes little time to refactor.

[paul-uz]

@richarddavison I;ve just done a quick test, replacing defaultProvider() with the env vars, and it looks like that works fine as a replacement.

I'll retry bundling this function for the LLRT and give it another go.

For future ref, how can I handle any 3rd party package that requires http/https? I think the docs talk about overrides, but the information is lacking.

[paul-uz]

So I'm using @smithy/node-http-handler to handle the response from the signed http request, and that is failing. What can I replace it with? Code below for context

const client = new NodeHttpHandler();

...


const signedRequest = await signer.sign(request);
const { response } = await client.handle(signedRequest as HttpRequest);
const finalResponse = await this.getFinalResponse(response);

getFinalResponse = async (response: HttpResponse): Promise<any> => {
  var responseBody = '';
  return await new Promise((resolve) => {
    response.body.on('data', (chunk: any) => {
      responseBody += chunk;
    });
    response.body.on('end', () => {
      resolve(JSON.parse(responseBody));
    });
  }).catch((error) => {
    console.error('Error: ' + error);
  });
};

[richarddavison]

For future ref, how can I handle any 3rd party package that requires http/https? I

This is a bit tricky. Depends on the third party package. Generally speaking, larger third party packages support bundling for browser which assume a fetch is used as http client. We are looking to bring in the lower level http & https packages clients to support more node packages.

So I'm using @aws-sdk/node-http-handler to handle the response from the signed http request, and that is failing. What can I replace it with? Code below for context

Use @smithy/fetch-http-handler.

Your code would be something like this then:

const client = new FetchHttpHandler();
...
const signedRequest = await signer.sign(request);
const { response } = await client.handle(signedRequest);
const str = JSON.parse(await response.Body.transformToString())

One thing to note though is that using the lower level APIs like this is not very common.

You probably have a very good reason for doing so but usually SDK integrations are done via the clients like shown in this example:
https://github.com/awslabs/llrt/blob/main/index.mjs

[paul-uz]

@richarddavison FYI const str = JSON.parse(await response.body.transformToString()) doesn't work.

[paul-uz]

You probably have a very good reason for doing so but usually SDK integrations are done via the clients like shown in this example:
https://github.com/awslabs/llrt/blob/main/index.mjs

Forgot to mention, in this case, its for OpenSearch. There is no AWS SDK for it, so we do it via a signed request. There is a OpenSearch NPM package I might look to move to, but i suspect it uses http/https and not fetch.

[paul-uz]

@richarddavison you'll be pleased to here that I got this function working finally.

Few tweaks here and there, and I've had to remove out internal SDK as that uses the node-http-handler.

Do you know if smithy/fetch-http-handler can be used even if we're not using the LLRT, but using Nodejs20 Lambdas?

[richarddavison]

Do you know if smithy/fetch-http-handler can be used even if we're not using the LLRT, but using Nodejs20 Lambdas?

Yes, you can! Works in node 20 as well!

[richarddavison]

Forgot to mention, in this case, its for OpenSearch. There is no AWS SDK for it, so we do it via a signed request. There is a OpenSearch NPM package I might look to move to, but i suspect it uses http/https and not fetch.

We do support OpenSearch:
https://www.npmjs.com/package/@aws-sdk/client-opensearch?activeTab=readme

We also support using fetch-http-handler for all SDKs. However, when using LLRT you can use the "full-sdk" release binary which includes OpenSearch client and don't worry about signing headers, credential providers, http handlers or parsing json from the raw response :)
https://github.com/awslabs/llrt/releases/download/v0.2.2-beta/llrt-lambda-arm64-full-sdk.zip

[paul-uz]

That client is for managing OpenSearch, not searching it ;)

[richarddavison]

That client is for managing OpenSearch, not searching it ;)

Ahhhh right, makes sense! I suppose I would try then just use fetch as is then and pass the sigv4 response to it directly:

const res = await fetch(signedResponse)
await res.json()