Feature/http client #1973

Jeroen88 · 2018-10-16T15:57:22Z

Add two begin() methods to HTTPClient() to pass a WiFiClientSecure client with setCACert() and/or setCertificate() and setPrivateKey() already set. Include an example BasicHttpsClient.ino to show the usage. This adaptation aligns this library with ESP8266HTTPClient. The 5 begin() methods that already existed are set deprecated and the library version is updated to 1.2.
Currently backwards compatibility is maintained, but the idea is to drop all code enclosed within #ifdef HTTPCLIENT_1_1_COMPATIBLE and #endif, when a new version of the core is released.
When using this library make sure that the passed in WiFiClientSecure 'lives' longer than the HTPPClient including it's destructor! In the example this is achieved by adding an extra scoping block { ... } for HTTPClient, to make sure it is destructed at the end of the block in stead of at the end of the function.

…() functions deprecated. Updated library version to 1.2

…estroyed before WiFiClientSecure

…Correct WiFiClient::setTimeout() to call Stream::setTimeout() with seconds converted back to milliseconds. Remove inproper checks for _insecure.

Jeroen88 · 2018-10-22T15:11:15Z

See PR #1979 for comments on setTimeout. This PR is included in #1979.

…inish

usrdes · 2018-11-25T05:26:25Z

Just updated my ESP32 install and was
attempting to use audio streaming in my Arduino (V1.8.6) sketch.
Now I start seeing HTTP issues, related to opening the URL. Before
the update the HTTP access was working fine.

Hope some one can help

The new HTTPClient "begin" routines seems to be doing something strange:
Trying to connect to the following URL:
'http://streaming.shoutcast.com/80sPlanet?lang=en-US'

However, other URL's also are broken up by beginInternal() and the access to the URL
fails. If I cut and paste the URL in browser, there is streaming audio. 
But it appears that the "80sPlanet" is being parsed as Port 80.

The key word "80sPlanet" is used to indicate 1980's music, I think, not sure,

Should I be inserting back_slash '\8\0s...' or some such escape chars, perhaps single quotes ?
Below is the verbose output on ESP32:
-----------
[V][HTTPClient.cpp:224] beginInternal(): url: http://streaming.shoutcast.com/80sPlanet?lang=en-US
[D][HTTPClient.cpp:265] beginInternal(): host: streaming.shoutcast.com port: 80 url: /80sPlanet?lang=en-US
[E][WiFiClient.cpp:236] setSocketOption(): 1006 : 9
[E][WiFiGeneric.cpp:659] hostByName(): DNS Failed for streaming.shoutcast.com
[D][HTTPClient.cpp:973] connect(): failed connect to streaming.shoutcast.com:80
[W][HTTPClient.cpp:1262] returnError(): error(-1): connection refused
[D][HTTPClient.cpp:369] disconnect(): tcp is closed


Another attempt with a different URL:

http://stream.srg-ssr.ch/m/rsj/mp3_128
-----------
[D][HTTPClient.cpp:208] begin(): mix up of new and deprecated api
Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.
Core 1 register dump:
PC      : 0x401eff46  PS      : 0x00060330  A0      : 0x800e0897  A1      : 0x3ffd1580  
A2      : 0x3ffe9268  A3      : 0x00000000  A4      : 0x3ffbaa8c  A5      : 0x00000000  
A6      : 0x00000000  A7      : 0xffffffbc  A8      : 0x8000beca  A9      : 0x3ffd1520  
A10     : 0xfefefefe  A11     : 0x3ffe87f0  A12     : 0x00000000  A13     : 0x00000001  
A14     : 0x00060520  A15     : 0x00000000  SAR     : 0x00000004  EXCCAUSE: 0x0000001c  
EXCVADDR: 0xfefefefe  LBEG    : 0x400014fd  LEND    : 0x4000150d  LCOUNT  : 0xfffffffe  

Backtrace: 0x401eff46:0x3ffd1580 0x400e0894:0x3ffd15a0 0x400e198a:0x3ffd15c0 0x400d981d:0x3ffd15f0 0x400d9955:0x3ffd1630 0x400d39e2:0x3ffd1650 0x400d400c:0x3ffd1670 0x401c77b6:0x3ffd16b0 0x40090cb5:0x3ffd16d0

Rebooting..

Jeroen88 · 2018-11-25T11:59:06Z

But it appears that the "80sPlanet" is being parsed as Port 80

No, port 80 is the default port for a http connection, this is added by the client because your url starts with http.

[E][WiFiGeneric.cpp:659] hostByName(): DNS Failed for streaming.shoutcast.com

This is the place where things go wrong

[D][HTTPClient.cpp:208] begin(): mix up of new and deprecated api

But this should not happen. This error appeared before, because _client was not inititialized to nullptr. This is corrected in PR #2097

Could you merge the latest core and next this PR and try again? Please report back if your issue was solved. If not, please create a new issue in stead of a comment in a PR

usrdes · 2018-11-26T04:30:22Z

@Jeroen88

Thank you for your response.

"Could you merge the latest core and next this PR and try again?"
I am using the release: Latest commit 273196d 7 days ago

This is still generating the issue.
This release was on 11/18, I did a complete install on 11/24.

For now, I have rolled back to the previous release and it appears to be working.
Not sure when I can get back and create a new issue.

Jeroen88 · 2018-11-26T06:08:43Z

@usrdes did you merge PR #2097 too? This is essential!

usrdes · 2018-11-26T07:24:59Z

@Jeroen88,
Sorry I do not know how to merge PR #2097. Is there any instructions on line?
Usually, I simply wipe out the installed ESP32 and re-install the latest release.

Thank you much for helping ..
Best

Jeroen88 · 2018-11-26T08:51:28Z

Do you re-install using git? Then you might read the instructions at e.g https://gist.github.com/adam-p/15413fabef6cffecd897

If you install using a zip file, it is probably easier to edit the file yourself and change it. In line 211 of file libraries/HTTPClient/src/HTTPClient.h you should change:

WiFiClient* _client;

into:

WiFiClient* _client = nullptr;

usrdes · 2018-11-26T16:41:26Z

@Jeroen88,

Thank you very much, I edited the file and that fixed it!
Thank you, Thank you

Jeroen88 · 2018-11-26T20:34:23Z

@usrdes You're welcome! If you reinstall in your way, your own change will get overwritten. Wait with reinstalling after PR #2097 is merged or re-edit the file.
UPDATE: the PR is just being merged.

marcelstoer · 2019-09-08T12:22:01Z

libraries/HTTPClient/examples/BasicHttpsClient/BasicHttpsClient.ino

+
+  Serial.print(F("Waiting for NTP time sync: "));
+  time_t nowSecs = time(nullptr);
+  while (nowSecs < 8 * 3600 * 2) {


Two questions:

Why not define (or calculate) this static value outside the loop?

Why exactly > 1.1.1970 4pm? Shouldn't it be larger than let's say 1546300800 (2019-01-01T00:00:00+00:00)? In my experience 16h is not enough.

@marcelstoer do you mean 8 * 3600 * 2 with 'this static value'? If so, you shouldn't bother, the compiler will replace it with a static value.

Basically the value is quite arbitrary, any big enough value will do. time(nullptr) will return the number of seconds since the last reset, until it is overwritten by the NTP time sync. After that time(nullptr) is far bigger than the static test value and the while loop exits.

time_t starts counting at the unix Epoch which you can read up about here.

If the time sync takes more than 16 hours there is a serious issue with network connectivity and time sync!

do you mean 8 * 3600 * 2 with 'this static value'?

Errhm, yes. Sorry. I should have said "constant" when I mean "constant".

the compiler will replace it with a static value

True, the comment was more a style nitpick (and in that quit subjective I know).

@Jeroen88 @atanisoft for my second comment I was drawing from ESP8266 (i.e. NONOS SDK) experience, maybe it doesn't apply here anymore. For devices that spent most of their life in deep sleep we had to ensure that the constant value was a somewhat recent timestamp - at least from the current century. Otherwise, we might get nowSecs values that are way off. while loops a handful of times. Hence, it has nothing to do with "time sync takes more than 16 hours" or so.

@marcelstoer I do not have any experience with deep sleep, but if time(nullptr) keeps running in deep sleep, the while() will break inmediately. I can imagine that in that case the ESP time and the real (network / NTP time) have drifted, and that you do want to have a time sync. The only thing I can think of here is to force a NTP time sync.

@atanisoft I didn't test it, but a freshly booted ESP32 could only initiate it's time after it has connected to the network, I guess? So some kind of waiting should be introduced to ensure synchronization has finished?

Jeroen88 added 4 commits October 14, 2018 10:11

Pass client parameter into two new begin() functions. Set other begin…

36a9a5a

…() functions deprecated. Updated library version to 1.2

Added working HTTPS example on a public url with a certificate

03c4da8

Remove two unnecessary tests in ::disconnect()

baeb171

Add a scoping block to BasicHttpsClient.ino to assure HTTPClient is d…

d7293a2

…estroyed before WiFiClientSecure

Jeroen88 mentioned this pull request Oct 18, 2018

Feature/http update #1979

Merged

Jeroen88 added 2 commits October 21, 2018 13:16

Added check to handle mixup of old and present api properly

367706e

Correct HTTPClient::setTimeout() to convert milliseconds to seconds. …

8275243

…Correct WiFiClient::setTimeout() to call Stream::setTimeout() with seconds converted back to milliseconds. Remove inproper checks for _insecure.

Added small comment because it looked like the Travis build did not f…

79724d6

…inish

me-no-dev merged commit 01d22c8 into espressif:master Nov 19, 2018

marcelstoer reviewed Sep 8, 2019

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature/http client #1973

Feature/http client #1973

Jeroen88 commented Oct 16, 2018

Jeroen88 commented Oct 22, 2018

usrdes commented Nov 25, 2018 •

edited

Loading

Jeroen88 commented Nov 25, 2018

usrdes commented Nov 26, 2018

Jeroen88 commented Nov 26, 2018

usrdes commented Nov 26, 2018 •

edited

Loading

Jeroen88 commented Nov 26, 2018

usrdes commented Nov 26, 2018

Jeroen88 commented Nov 26, 2018 •

edited

Loading

marcelstoer Sep 8, 2019

Jeroen88 Sep 8, 2019

atanisoft Sep 8, 2019

marcelstoer Sep 8, 2019

Jeroen88 Sep 8, 2019

Feature/http client #1973

Feature/http client #1973

Conversation

Jeroen88 commented Oct 16, 2018

Jeroen88 commented Oct 22, 2018

usrdes commented Nov 25, 2018 • edited Loading

Jeroen88 commented Nov 25, 2018

usrdes commented Nov 26, 2018

Jeroen88 commented Nov 26, 2018

usrdes commented Nov 26, 2018 • edited Loading

Jeroen88 commented Nov 26, 2018

usrdes commented Nov 26, 2018

Jeroen88 commented Nov 26, 2018 • edited Loading

marcelstoer Sep 8, 2019

Choose a reason for hiding this comment

Jeroen88 Sep 8, 2019

Choose a reason for hiding this comment

atanisoft Sep 8, 2019

Choose a reason for hiding this comment

marcelstoer Sep 8, 2019

Choose a reason for hiding this comment

Jeroen88 Sep 8, 2019

Choose a reason for hiding this comment

usrdes commented Nov 25, 2018 •

edited

Loading

usrdes commented Nov 26, 2018 •

edited

Loading

Jeroen88 commented Nov 26, 2018 •

edited

Loading