[SOLVED] U97 SOAP call-out to https site got error

Hi, I haven't read your topic very carefully, but have you looked at this topic http://unifaceinfo.com/forum/uniface9/web-service-u9-7/ and especially this: For some reason I had to reimport the wsdl. I see no important change in the signature exported from U9.6.04 to U9.7 and the one reimported into U9.7 ?? The above indicates that I had to reimport the wsdl with "/sti", it wasn't enough to migrate the signature?? And the need for using a "local" file [SERVICES_EXEC] smsservice=$sop:smsservice  wsdl=g:/unifaceappdir/SMSService.wsdl  Regards RogerW.

Hi Marco, could be obvious but...considering https protocol: do you migrated related certificate as well ? Gianni

gianni said Hi Marco, could be obvious but...considering https protocol: do you migrated related certificate as well ? Gianni  

Hi Gianni, correct, but I haven't certificate in U94 and even in U97. I use ign=HP parameter on SOAP params in asn definition to remove control on there. In my case the error is on HTTP transport, but I tried to remove ign parameter and I receved a correct certificate error. I tried also to use a ca_bundle.crt downloaded from mozilla site, as suggested in other post, and tried different combination of ign parameter (only H, only P). Marco

Hi Marco, Do you know what web server your customer is using? And have you checked the complete response from the server in the SOAP_CALLOUT_POST operation? Maybe it contains some additional info about the error? I've seen the HTTP error 411 before with (e.g.) Apache version 1.3 and chunked transfer encoding. The workaround was to use the HTTP version 1.0 instead of 1.1. It could be that the version in the User-Agent header refers to the HTTP version used by curl ("CURLTransport/1.0" and "CURLTransport/1.1"). Not sure though. Unfortunately it is not possible to specify the HTTP version the SOAP connector should used. AFAIK it will always use the internally preferred HTTP version of curl. So in case HTTP version 1.0 is required then it might be an option to call the webservice by doing the request with the curl command line tool and spawn it from Uniface (or use the OS command signature). Hope this helps. Kind regards, Daniel Iseli Uniface Technical Support

Hi Daniel, I don't know what web server version is on the third part (not my customer) and the SOAP_CALLOUT_POST isn't executed after the activate command, only the SOAP_CALLOUT_PRE is executed. So, there isn't the solution by code? It's not a bug? In U94 the SOAP call-out with activate works, in U97 doesn't work anymore. As the problem is HTTP error 411 Length Required and as both UHTTP and SOAP call-out use cURL library, is possibile that the FLAG 4 (Do not calculate the content-length of the payload itself for a POST method), visible in UHTTP operation, is set by default on SOAP call-out? Or maybe the error is in the cURL library? In this case, replace libcurl.dll with the old version from U94 (that I tried and it works) is possible and correct? I tried to install the new cURL library (libcurl.dll W32 edition) downloaded from cURL site, but on activate I got a -81 procerror and the error context indicates "Failure to connect to engine".   Marco

Marco said Hi Daniel,

Hi Marco,

Marco said I don't know what web server version is on the third part (not my customer) and the SOAP_CALLOUT_POST isn't executed after the activate command, only the SOAP_CALLOUT_PRE is executed.

Indeed, in case the returned HTTP status is 411 (or anything other than 200) then the SOAP_CALLOUT_POST operation is not activated. We could e.g. use a web proxy (like e.g. Fiddler) or the command line tool curl (Uniface 9.7.03 + G308 is e.g. using the version 7.40.0) to figure out the exact reply from the webservice.

Marco said So, there isn't the solution by code? It's not a bug? In U94 the SOAP call-out with activate works, in U97 doesn't work anymore.

Currently I can't think of a code solution and I'm also not convinced that this is a bug. I did a quick check and my assumption seems to be correct that the Uniface 9.4 SOAP connector (i.e. libcurl) is using HTTP version 1.0 and Uniface 9.7 HTTP version 1.1. So it's quite possible that the web server has a problem with chunked transfer encoding and that's the reason for the HTTP error 411. A possible workaround could be to use the curl command line tool and use HTTP version 1.0 (see command line option -0, --http1.0).

Marco said As the problem is HTTP error 411 Length Required and as both UHTTP and SOAP call-out use cURL library, is possibile that the FLAG 4 (Do not calculate the content-length of the payload itself for a POST method), visible in UHTTP operation, is set by default on SOAP call-out?

No, I don't think so. The SOAP connector will add the Content-Length header to the request.

Marco said Or maybe the error is in the cURL library? In this case, replace libcurl.dll with the old version from U94 (that I tried and it works) is possible and correct? I tried to install the new cURL library (libcurl.dll W32 edition) downloaded from cURL site, but on activate I got a -81 procerror and the error context indicates "Failure to connect to engine".

See above - I don't think that there's an error in the libcurl.dll. And please don't copy the libcurl.dll from an older Uniface version or try to replace it with another curl version. The libcurl.dll used by Uniface is modified and replacing it with something else could break the SOAP connector or might make Uniface unstable.

Marco said   Marco  

Hope this helps. Daniel

Hi Roger, I tried your suggestion, but it doesn't work, I got same error. I tried to launch curl 7.40.0 command line and it works perfectly, so why libcurl.dll inside U97 doesn't work? This is the line that I run: curl --ntlm --insecure --user userid:password --header "Content-Type: text/xml;charset=UTF-8" --header "SOAPAction:http://XXXXXXXXXXXXX/XXXXXXXXXXXXXXXXX" --data @envelope.xml https://xxxxxxxx where envelope.xml is catched with SOAP_CALLOUT_PRE function. I hope that the switches used are like $SOP parameter used in asn file (scheme=L ign=HP). With verbose option, I saw that the server http protocol is 1.1. This is the result:

  Trying 111.111.111.111... * Connected to XXXXXXXXXXXXXXXXX (111.111.111.111) port 443 (#0) * TLSv1.0, TLS handshake, Client hello (1): } [199 bytes data] * TLSv1.0, TLS handshake, Server hello (2): { [81 bytes data] * TLSv1.0, TLS handshake, CERT (11): { [2818 bytes data] * TLSv1.0, TLS handshake, Server finished (14): { [4 bytes data] * TLSv1.0, TLS handshake, Client key exchange (16): } [262 bytes data] * TLSv1.0, TLS change cipher, Client hello (1): } [1 bytes data] * TLSv1.0, TLS handshake, Finished (20): } [16 bytes data] * TLSv1.0, TLS change cipher, Client hello (1): { [1 bytes data] * TLSv1.0, TLS handshake, Finished (20): { [16 bytes data] * SSL connection using TLSv1.0 / AES128-SHA * Server certificate: *        subject: XXXXXXXXXXXXXXXXX *        start date: 2017-02-16 00:00:00 GMT *        expire date: 2019-03-18 23:59:59 GMT *        issuer: C=US; O=GeoTrust Inc.; CN=GeoTrust SSL CA - G3 *        SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway. * Server auth using NTLM with user 'XXXXXXXXXXX' > POST /service.asmx HTTP/1.1 > Authorization: NTLM TlRMTVNTUAABAAAAB4IIogAAAAAAAAAAAAAAAAAAAAAGAbEdAAAADw== > User-Agent: curl/7.40.0 > Host: XXXXXXXXXXXXXXXXX > Accept: */* > Content-Type: text/xml;charset=UTF-8 > SOAPAction:http://XXXXXXXXXXXXX/XXXXXXXXXXXXXXXXX > Content-Length: 0 > < HTTP/1.1 401 Unauthorized < Content-Type: text/html; charset=us-ascii < Server: Microsoft-HTTPAPI/2.0 < WWW-Authenticate: NTLM TlRMTVNTUAACAAAACAAIADgAAAAFgomi0UR/dcIDUDgAAAAAAAAAAJ4 < Date: Mon, 12 Jun 2017 14:00:40 GMT < Content-Length: 341 < * Ignoring the response-body { [341 bytes data] * Connection #0 to host XXXXXXXXXXXXXXXXXX left intact * Issue another request to this URL: 'https://xxxxxxxxxxxx * Found bundle for host XXXXXXXXXXXXXXXXX: 0x87d798 * Re-using existing connection! (#0) with host XXXXXXXXXXXXXXXXXXXXXX * Connected to XXXXXXXXXXXXXXXX (111.111.111.111) port 443 (#0) * Server auth using NTLM with user 'XXXXXXXXXXXXXXXX' > POST /service.asmx HTTP/1.1 > Authorization: NTLM TlRMTVNTUAADAAAAGAAYAIgAAAAmASYBoAAAAAgACABYAAAAHAAcAGAAAA > User-Agent: curl/7.40.0 > Host: XXXXXXXXXXXXXXXXXXXXXX > Accept: */* > Content-Type: text/xml;charset=UTF-8 > SOAPAction:http://XXXXXXXXXXXXX/XXXXXXXXXXXXXXXXX > Content-Length: 713 > } [713 bytes data] * upload completely sent off: 713 out of 713 bytes < HTTP/1.1 200 OK < Cache-Control: private, max-age=0 < Content-Type: text/xml; charset=utf-8 < Server: Microsoft-IIS/7.5 < X-AspNet-Version: 2.0.50727 < Persistent-Auth: true < X-Powered-By: ASP.NET < Date: Mon, 12 Jun 2017 14:00:42 GMT < Content-Length: 703748 < { [16128 bytes data] * Connection #0 to host XXXXXXXXXXXXXXXXX left intact

   Marco

Hi Marco, Thanks for the additional info. This is really helpful. From the curl tracing I can see that the Web Service is Microsoft IIS 7.5. I did a quick test here on one of our Windows Test-Servers and I can see the problem now as well. It seems that when the SOAP connector is sending the initial request with the NTLM authorization then the Content-Length header is not set (and this seems to trigger the HTTP error 411). It looks like that this issue was introduced after we've upgraded the curl version used by Uniface to 7.40.0 (with the version 9.6.06 patch X604 and version 9.7.01). Before the curl upgrade the SOAP connector sets the Content-Length header to 0 ("Content-Length: 0") and after the upgrade the header is omitted. I'll discuss this with the lab and shall keep you posted. The good news is that I've found a workaround: when you set the scheme connector option to LN (NTLM and Negotiate) then the request should work (again) correctly. E.g.

[SERVICES_EXEC]
GAC_ARTICOLI            $SOP:GAC_ARTICOLI euser=domain\user epass=password scheme=LN ign=HP

  Hope this helps. Daniel

Hi Daniel, I tried the workaround and now SOAP call-out works correctly again. Thanks Marco

Marco said Hi Daniel, I tried the workaround and now SOAP call-out works correctly again. Thanks Marco  

Thanks, Marco. It's good to hear that the workaround is solving the problem. In the meantime I had a chat with one of the engineers in the lab. It seems that curl has a problem when the Context-Length header is explicitly set (which the SOAP connector does for all requests) and NTLM authentication is used. curl also seems to add the mentioned header (i.e. Content-Length: 0) and when it already has been set by the calling code (i.e. the SOAP connector) then this will result in the Context-Length header being removed from the request. This was not an issue with older curl versions (at least before version 7.40). The workaround for this issue is to omit the Content-Length header from the request (since it automatically will be added by curl). I'll create a new bug report for this issue. But I don't think that this will be resolved quickly, since there is an easy workaround for this issue (i.e. set scheme=LN). Hope this helps. If you have any further questions or concerns regarding this issue then let me know. Daniel

Just a quick update: I've created in the meantime, as promised, a new BUG report for this problem:

• BUG: 31620 - SOP U2.0: WS Call-Out fails when authentication scheme NTLM is used

This issue will be added to the maintenance backlog with a low priority. In case this problem is time critical for someone then please log a call with support. Hope this helps. Daniel

Just another update: the BUG#31620 just has been fixed with the patch G413. Hope this helps. Daniel