Error build/link against specified curl when system installed curl found

[ChuckCottrill]

Describe the bug

Error building aws-sdk-ccp-core
want to link against local download and built curl-8.0.1
The variables CURL_INCLUDE_DIR, CURL_LIBRARY, are specified,
- CURL_INCLUDE_DIR="${DEPEND_INSTALL_DIR}/curl-${LIBCURL_VERSION}/include" \
- CURL_LIBRARY="${DEPEND_INSTALL_DIR}/curl-${LIBCURL_VERSION}/lib/libcurl.dylib" \
These desired directories exist and are correctly referenced
But cmake files look for system libraries first, using FindCURL.

Expected Behavior

Curl library and include directories, CURL_INCLUDE_DIR, CURL_LIBRARY, should be recognized and used to access specified version of curl. Or, an option should be provided to configure cmake to use these variables instead of system values.
Linking should succeed.

Current Behavior

message:
CMake Warning:
Manually-specified variables were not used by the project:

  CURL_INCLUDE_DIR
   CURL_LIBRARY
...
[ 81%] Linking CXX shared library libaws-cpp-sdk-core.dylib
Undefined symbols for architecture arm64:
 "_curl_easy_cleanup", referenced from:
     Aws::Http::CurlHandleContainer::~CurlHandleContainer() in CurlHandleContainer.cpp.o
     Aws::Http::CurlHandleContainer::DestroyCurlHandle(void*) in CurlHandleContainer.cpp.o
 "_curl_easy_getinfo", referenced from:
     Aws::Http::CurlHttpClient::MakeRequest(std::__1::shared_ptr<Aws::Http::HttpRequest> const&, Aws::Utils::RateLimits::RateLimiterInterface*, Aws::Utils::RateLimits::RateLimiterInterface*) const in CurlHttpClient.cpp.o
...
 "_curl_version_info", referenced from:
     Aws::Http::CurlHttpClient::InitGlobalState() in CurlHttpClient.cpp.o
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [aws-cpp-sdk-core/libaws-cpp-sdk-core.dylib] Error 1
make[1]: *** [aws-cpp-sdk-core/CMakeFiles/aws-cpp-sdk-core.dir/all] Error 2

Reproduction Steps

Have local version of curl installed in /usr/local/lib, with /usr/local/lib/cmake/CURL directory

CC=/usr/bin/clang CXX=/usr/bin/clang++ cmake \
    -DCMAKE_CXX_FLAGS="-fstack-protector-strong -Wno-deprecated-declarations" \
    -DCMAKE_C_FLAGS="-fstack-protector-strong -Wno-deprecated-declarations" \
    -DBUILD_SHARED_LIBS=ON \
    -DCURL_INCLUDE_DIR="${DEPEND_INSTALL_DIR}/curl-${LIBCURL_VERSION}/include" \
    -DCURL_LIBRARY="${DEPEND_INSTALL_DIR}/curl-${LIBCURL_VERSION}/lib/libcurl.dylib" \
    -DCMAKE_PREFIX_PATH="${DEPEND_INSTALL_DIR}/aws-sdk-${AWS_SDK_CPP_VERSION}" \
    -DCMAKE_INSTALL_PREFIX="${DEPEND_INSTALL_DIR}/aws-sdk-${AWS_SDK_CPP_VERSION}" \
    -DBUILD_ONLY="ebs" \
    -DCPP_STANDARD="${CPP_STANDARD}" \
    -DENABLE_TESTING=OFF \
    -S ${DEPEND_BUILD_DIR}/aws-sdk-${AWS_SDK_CPP_VERSION}/aws-sdk-cpp \
    -B ${DEPEND_BUILD_DIR}/aws-sdk-${AWS_SDK_CPP_VERSION}/aws-sdk-cpp-build

These directories exist and are accessible,

CURL_INCLUDE_DIR="${DEPEND_INSTALL_DIR}/curl-${LIBCURL_VERSION}/include" \
CURL_LIBRARY="${DEPEND_INSTALL_DIR}/curl-${LIBCURL_VERSION}/lib/libcurl.dylib" \

Possible Solution

One option could be provided to configure cmake to use/prefer these variables instead of system values.

• CURL_INCLUDE_DIR
• CURL_LIBRARY

Another option would be to add a flag (e.g. CURL_USE_SYSTEM or CURL_USE_SPECIFIED) to indicate which library to prefer.

Additional Information/Context

There is a system-wide setting, NO_CMAKE_SYSTEM_PATH, but that prevents finding any system installed libraries.

AWS CPP SDK version used

1.9.379

Compiler and Version used

clang++ --version Apple clang version 14.0.3 (clang-1403.0.22.14.1)

Operating System and version

mac os ventura

[yasminetalby]

Hello @ChuckCottrill ,

Thank you very much for your submission.
It seems like this behavior has been reported in the past : #938 with potential workaround for your issue. I'll bring this up to the team today.

Thank you for your time and collaboration,

Sincerely,

Yasmine

[yasminetalby]

Hello @ChuckCottrill ,

Could you try passing -DCMAKE_PREFIX_PATH={path to openssl where lib and include dirs are found} and let me know if this resolves your issue. Currently, the support for CURL_INCLUDE_DIR and CURL_LIBRARY would be something we would consider as part of #1888 .

Best,

Yasmine

[ChuckCottrill]

Hello Yasmine,
Thank you for your suggestion. We already use CMAKE_PREFIX_PATH to provide location of AWS_SDK:
-DCMAKE_PREFIX_PATH="${DEPEND_INSTALL_DIR}/aws-sdk-${AWS_SDK_CPP_VERSION}"''

I would be happy to try the change you suggested.
Should I add the CURL library path after the AWS_SDK path using a colon (":")?

[SergeyRyabinin]

Hi @ChuckCottrill ,

I'm curious why you would need to set CMAKE_PREFIX_PATH to the location of the SDK when you are building the SDK itself.
Typically it is a variable that is used to specify the location of dependencies
The last time I built the SDK with a custom libcurl, I used the following command:

cmake -B build -S . \
      -DBUILD_ONLY="s3;s3-crt" \
      -DCMAKE_INSTALL_PREFIX=$HOME/sdk-install \
      -DCMAKE_PREFIX_PATH=$HOME/curl-install

cmake --build build -j 8 --config Debug --target install

and then built the sample application that is going to use built and installed SDK:

cmake -B build_sample_app -S ../tools/CI/install-test -DCMAKE_PREFIX_PATH=$HOME/sdk-install
cmake --build build_sample_app -j 8 --config Debug

Should I add the CURL library path after the AWS_SDK path using a colon (":")?

According to cmake documentation, CMAKE_PREFIX_PATH is a Semicolon-separated list of directories specifying installation prefixes to be searched by the find_package()...
So we need to use semicolon ; to separate them, please note that ; can be also interpreted by bash/zsh, so we need to add quotes, for example, if I want to build an app that uses the SDK and libcurl (independently from the SDK):

 -DCMAKE_PREFIX_PATH="$HOME/sdk-install;$HOME/curl-install"

Best regards,
Sergey

[github-actions]

Greetings! It looks like this issue hasn’t been active in longer than a week. We encourage you to check if this is still an issue in the latest release. Because it has been longer than a week since the last update on this, and in the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or add an upvote to prevent automatic closure, or if the issue is already closed, please feel free to open a new one.