HttpCommand

Repository	https://github.com/dyalog/httpcommand/
Copyright	Copyright © 2015-2022 Dyalog, LTD

Note: While HttpCommand itself is ⎕IO and ⎕ML insensitive, the examples in this documentation assume an environment of (⎕IO ⎕ML)←1.

HttpCommand is a utility is designed to make it easy for the APL user to send requests to and receive responses from HTTP servers like web servers and web services. HttpCommand is included with Dyalog APL as a loadable utility. To bring it into your workspace, simply do:

      ]Load HttpCommand

or, under program control, do:

     ⎕SE.SALT.Load 'HttpCommand'

Beginning with Dyalog v18.2, you can also use the ]get user command:

      ]Get HttpCommand

Upgrading to `HttpCommand` Version 5

This documentation describes version 5 of HttpCommand. Dyalog versions 18.2 and 18.0 have earlier versions of HttpCommand pre-installed. To upgrade your in-workspace copy of HttpCommand to version 5, you may have to run HttpCommand.Upgrade once or twice depending on how recent your Dyalog installation is.

If HttpCommand.Version reports a version earlier than 4.0.14, HttpCommand.Upgrade will upgrade to the latest version 4. Then running HttpCommand.Upgrade once more will upgrade to version 5.

For example, HttpCommand version 3.4 may be found in Dyalog version 18.0.

       ]load HttpCommand
#.HttpCommand
      HttpCommand.Version
 HttpCommand  3.4  2021-05-28 
      HttpCommand.Upgrade
0   
      HttpCommand.Version  ⍝ verify that we upgraded to v4.0.14
 HttpCommand  4.0.14  2022-09-04 
      HttpCommand.Upgrade  ⍝ now upgrade to v5
0  Upgraded to HttpCommand 5.0.7 2022-08-27 from HttpCommand 4.0.14 2022-09-04

It is recommended that you save your own copy of HttpCommand in your application rather than dynamically getting or upgrading it at runtime.

Typical Usage Pattern

HttpCommand is implemented as a Dyalog class and its typical usage pattern is:

Create an instance of the HttpCommand class using the function HttpCommand.New
instance ← HttpCommand.New ''
or by using the ⎕NEW system function
instance ← ⎕NEW HttpCommand
Set fields (settings) appropriate to describe the request
Send the request
Examine the response result

For example:

      cmd←HttpCommand.New ''   ⍝ create an instance
      cmd.Command←'get'        ⍝ specify the HTTP method 
      cmd.URL←'www.dyalog.com' ⍝ specify the URL

      ⊢response←cmd.Run        ⍝ send the request
[rc: 0 | msg: "" | HTTP Status: 200 "OK" | ⍴Data: 21060]

      50↑response.Data         ⍝ examine the response payload
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Trans

The result of HttpCommand.Run is a namespace containing information about the response from server. The display format for the namespace gives some useful information about the success of the request. The server's response data, also known as the content or payload, is found in the Data element of the namespace.

HttpCommand has several "shortcut" shared methods to perform common operations. For instance, the above example can be accomplished by using HttpCommand.Get.

      50↑(HttpCommand.Get 'www.dyalog.com').Data
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Trans

Philosophy

HttpCommand has been designed to be easy to integrate into your application by having a small footprint (it's a single class) and making almost no assumptions about the environment in which you'll use it. HttpComand attempts to use sensible default values for request settings while still giving you full control should you need it. HttpCommand's feature set has expanded over time as new use cases and feature requests have arisen. We expect this trend to continue and encourage you to submit feature requests or report any bugs via GitHub.

This documentation should provide sufficient information to use HttpCommand to make most HTTP requests. In some cases, some further knowledge of HTTP may be necessary. This documentation does not attempt to be an exhaustive HTTP reference.

Dyalog Classes

HttpCommand is implemented as a Dyalog class. While it's not necessary to have an in-depth knowledge of Dyalog classes, it is useful to understand a couple basic concepts. A class can have "shared" methods and settings (known as fields) and "instance" methods and settings. For instance, the "shortcut" methods like Get, GetJSON, Do and New are shared methods whereas the instance methods like Show and Run are only available in an HttpCommand instance. Similarly, Debug and CongaPath are shared settings whereas URL and Command are instance settings. We will attempt to make the distinction between shared and instance references throughout this documentation.

Use of Conga

HttpCommand uses Conga, Dyalog's TCP/IP utility library, for communication. Conga is installed as a part of Dyalog APL. By default HttpCommand will attempt to locate and use the installed Conga. Conga consists of two parts:

Two shared library files that are specific to the operating system and implement the low-level communications and security interfaces.
An APL object that implements a cross-platform interface to the shared libraries. The object will be either the DRC namespace or an instance of the Conga.LIB class. For the purposes of this document we will refer to this object as the Conga API.

See HttpCommand and Conga for more information on how HttpCommand interacts with Conga. See Intergrating HttpCommand for more information on how to package Conga and HttpCommand in a distributed application.

Usage ↵

Shortcut Methods

HttpCommand has several shared "shortcut" methods to perform common operations. An overview of these methods is presented here. See the Reference section of the documentation for detailed information on all features of HttpCommand.

HttpCommand's Get, GetJSON, Do, and New methods each accept a right argument which is one of:

a vector of positional arguments
a namespace containing named arguments.

Positional arguments must be supplied in the position and order shown. If you do not need to use an intermediate argument, you should supply ''. For instance, a request that needs a Headers setting, but does not use a Params setting would be:

      HttpCommand.Get 'www.someurl.com' '' ('accept' 'image/png')

To use a namespace argument, create a namespace and assign the appropriate parameters for the request. For example:

      ns←⎕NS ''
      ns.(Command URL)←'post' 'www.someurl.com'
      ns.Params←⎕JSON ⍳10
      ns.Headers←'Content-Type' 'application/json'
      r←HttpCommand.Do ns

`Get`

Sends an HTTP request using the HTTP GET method.
r← [ro] HttpCommand.Get URL [Params [Headers]]
r← [ro] HttpCommand.Get namespace

`GetJSON`

Sends an HTTP request to a JSON-based web service using the HTTP method you specify. Unless otherwise specified, request parameters and response payload are automatically converted between APL and JSON.
r← [ro] HttpCommand.GetJSON Command URL [Params [Headers]]
r← [ro] HttpCommand.GetJSON namespace

`Do`

Sends an HTTP request using the HTTP method you specify.
r← [ro] HttpCommand.Do Command URL [Params [Headers]]
r← [ro] HttpCommand.Do namespace

`New`

Creates a new instance of HttpCommand.
h← [ro] HttpCommand.New Command URL [Params [Headers]]
h← [ro] HttpCommand.New namespace

Once the instance is created, additional settings can be assigned directly if necessary. Then use the Run method to send the request.
r← h.Run

Arguments

Name	Description	Default Value
`Command`	The HTTP method to use	`'GET'`
`URL`	The URL to which the request will be sent. If `URL` begins with `'https://'` `HttpCommand` will attempt create a secure connection to the host using either an anonymou)s certificate it creates or `Cert` if it is supplied	`''`
`Params`	Any parameters that are to be sent with the request.	`''`
`Headers`	Any HTTP headers (name/value pairs) to send with the request	`''`
`ro`	"Request Only" - when set to 1, `HttpCommand` will compose and return the HTTP request that it would send to the host, but not actually send it. This is useful for debugging to ensure your request is properly formed.	`0`

See the Request-related Settings section for more detail on the arguments.

Each of the above methods also accept 3 additional positional arguments, [Cert [SSLFlags [Priority]]]. These arguments are used to specify settings for secure connections but are seldom used in practice because HttpCommand automatically supplies an anonymous certificate when using a secure connection. See Secure Connections and Conga-related Settings for more information.

Shortcut or Instance?

There are two ways to use HttpCommand.

Use one of its shortcut methods (Get, Do, GetJSON) to execute a single HTTP request and examine the response from the host. When you use a shortcut method, HttpCommand will create a new, local, instance of itself and then use the instance's Run method to send the request and receive the response, then exit thereby destroying the local instance. This is generally fine when your requests are independent of one another and do not require any state to be maintained across requests.
Create an instance of HttpCommand using ⎕NEW or the New shared method, set whatever parameters are necessary, execute the Run method, and examine the response from the host. Creating an instance of HttpCommand and using the Run method to send the request and receive the response will persist the instance such that state information like connection information and HTTP cookies will be retained across requests.

`HttpCommand`'s Result

The result of the Run method is a namespace containing information related to the result of the request. The display format (⎕DF) of the result presents some useful information.

      ⊢r ← HttpCommand.Get 'https://www.dyalog.com'
[rc: 0 | msg:  | HTTP Status: 200 "OK" | ⍴Data: 20571]

r is the namespace result
r.rc is HttpCommand's numeric return code. 0 means HttpCommand was able to create and send the request to the host, send it, and receive and process the response.
If rc is less than 0, it means there was some problem either composing/sending the request or processing the response.
If rc is greater than 0, it is the Conga return code and generally means that Conga encountered something unexpected. More information about rc can be found here.
r.msg is a (hopefully meaningful) message describing whatever the issue was if r.rc is non-zero.
r.HttpStatus is the numeric HTTP status code returned by the host. A status code in the range 200-299 indicates a successful HTTP response with 200 being the most common code.
r.HttpMessage is the HTTP status message returned by the host.
r.Data is the payload of the response, if any, returned by the host. The result namespace contains other elements which are described in detail on the Request Results, Operational Results, and Response Results pages.

Typical Use in Code

Typical use of HttpCommand might follow this pattern.

 resp ← HttpCommand.Get 'some-url'
:If 0 200 ≢ resp.(rc HttpStatus)
    ⍝ code to handle bad request
:Else
    ⍝ code to process the response 
:EndIf

If you expect to make several HttpCommand calls, you may want to create an instance and then update the necessary settings and execute the Run method for each call. This is particularly useful when the requests are made to the same host as the connection to the host will be kept open, unless the host itself closes it.

 hc←HttpCommand.New 'get'  ⍝ here we expect all requests to be HTTP GET
 urls←'url1' 'url2' 'url3'

:For url :In urls ⍝ loop over the URLs
  hc.URL←url ⍝ set this URL
  resp←hc.Run
  :If 0 200≡resp.(rc HttpStatus)
    ⍝ process the response
  :Else
    ⍝ process the exception/error
  :EndIf
:EndFor

Interacting with Web Services

A web service will generally have a published API which describes how to interact with the service. This API will be your guide for how to build requests using HttpCommand to access the web service. In many cases, the API will also describe the format of the data it returns in the content-type header.

A web service may require authentication in order to gain access to certain resources or perform certain operations. GitHub is a good example of this - it does not require authentication to perform read-only queries on public resources, but authentication is necessary to perform write operations or to access private repositories.

Let's look at GitHub's REST API as an example. The GitHub API returns its data in JSON format.

Note: If you run these same examples, you may get slightly different results because of updates to Dyalog's content on GitHub.

      c ← HttpCommand.New 'get' 'https://api.github.com/orgs/Dyalog'

      ⊢r ← c.Run ⍝ send the request
[rc: 0 | msg:  | HTTP Status: 200 "OK" | ⍴Data: 1101]

      r.Headers c.Lookup 'content-type' ⍝ check the content-type
application/json; charset=utf-8

      d←⎕JSON r.Data ⍝ convert the JSON 

      d.(name public_repos) 
 Dyalog  48

      html ← 'HTML' ('<img src="',d.avatar_url,'"/>')
      coord ← 'coord' 'pixel' ⋄ size ← 'size' (250 250)
      'h' ⎕WC 'HTMLRenderer' html coord size

Authentication/Authorization

The example above uses no authentication and therefore returns only publicly-available information. Many web services require some form of authentication to provide authorization to access private resources or functionality. Two common authentication schemes are "token-based" and "HTTP Basic".

HTTP Basic Authentication

Some web services allow you to use HTTP Basic authentication using userid and password credentials. Any request that includes authentication data should be sent using HTTPS if possible. HttpCommand allows you to specify these credentials in a few ways:

in the URL by including userid:password@ before the domain name
by setting AuthType to 'basic' and
Auth to HttpCommand.Base64Encode 'userid:password'
by including an "Authorization" header in the format
'basic ',HttpCommand.Base64Encode 'userid:password'

The following examples use the "basic-auth" endpoint of the website http://httpbin.org. httpbin.org provides many useful endpoints for testing HTTP requests. In this case, the userid and password are concatenated to the URL so the endpoint can validate the credentials.

First, let's try using credentials in the URL...

      u ← 'userid'
      p ← 'password'
      endpoint ← 'httpbin.org/basic-auth/userid/password'

      ⊢url ← 'https://',u,':',p,'@',endpoint
https://userid:password@httpbin.org/basic-auth/userid/password

      ⊢r ← HttpCommand.Get url 
[rc: 0 | msg:  | HTTP Status: 200 "OK" | ⍴Data: 49]
      r.Data
{
  "authenticated": true, 
  "user": "user"
}

If we alter the URL so that the credentials don't match, we'll get a 401 "UNAUTHORIZED" HTTP status.

      HttpCommand.Get url,'1'
[rc: 0 | msg:  | HTTP Status: 401 "UNAUTHORIZED" | ⍴Data: 0]

Now, let's try setting the "Authorization" header...

      ⊢credentials ← HttpCommand.Base64Encode u,':',p
dXNlcmlkOnBhc3N3b3Jk
      hdrs ← 'Authorization' ('basic ',credentials)
      url ← 'https://',endpoint
      HttpCommand.Get url '' hdrs
[rc: 0 | msg:  | HTTP Status: 200 "OK" | ⍴Data: 49]

Finally, let's use the AuthType and Auth settings...

      h ← HttpCommand.New 'get' url ⍝ create an instance
      h.Auth ← credentials
      h.AuthType ← 'basic'
      h.Run
[rc: 0 | msg:  | HTTP Status: 200 "OK" | ⍴Data: 49]

Token-based Authentication

One common authentication method uses tokens which are issued by the web service provider. Tokens should be protected just as you would protect your password. A token may have an expiration date after which you'll need to get a new token. Any request that includes authentication data should be sent using HTTPS if possible.

For this next example we'll use a GitHub Personal Access Token which happens to provide more access to Dyalog's information on GitHub.

The API documentation for the web service you are using should provide information about the use of tokens. In GitHub's case, the token is passed in the HTTP request's "Authorization" header. HttpCommand allows you to specify the token either by creating an "Authorization" header or by setting the Auth configuration setting. If you set the Auth setting, HttpCommand will add the "Authorization" header automatically.

Let's assume that the token we're using in this example is in a variable named token. (Obviously, we are not going to publish the actual token here.) Also note if you plan on making several requests to the same web service, it's easier to create an instance of HttpCommand, set the Auth configuration setting, then set any other configuration settings and run the Run method for each request.

First, let's compare the difference between an authorized and a non-authorized request...

      ⊢HttpCommand.Get 'https://api.github.com/orgs/Dyalog'
[rc: 0 | msg:  | HTTP Status: 200 "OK" | ⍴Data: 1101]

      hdr←'authorization' ('token ',token)

      ⊢HttpCommand.Get 'https://api.github.com/orgs/Dyalog' '' hdr
[rc: 0 | msg:  | HTTP Status: 200 "OK" | ⍴Data: 1804]

Note more data is returned for the authorized request. This additional data will include things like organizational and administrative information.

Interacting with JSON-based Web Services

Many web services including those based on Jarvis, Dyalog's web service framework, use JSON to format their request and response payloads. Normally, to interact with a JSON-based web service, you would set the "content-type" header of the request to "application/json" if the request has a payload; then you would process the response's JSON payload, possibly with ⎕JSON.

HttpCommand's GetJSON method is designed to make it easy to to interact with such services. GetJSON will convert an APL array request payload to JSON format and conversely convert a JSON response payload to APL. We saw in the Interacting with Web Services example above that GitHub returns its result as JSON.

Posting Data to a Web Service

So far we've only read data from GitHub using the HTTP GET method. The GitHub API also allows authenticated users to create and update information. In this example we will create a new repository using the HTTP POST method and our GitHub Personal Access Token. For security reasons, we will use the same token variable from the previous example. At the time of this writing, we are using the GitHub API documentation for creating a repository for an authenticated user as a guide.

For this example, we'll use a namespace, args, to specify all the settings for our request...

      args←⎕NS ''
      args.Command            ← 'post'
      args.URL                ← 'https://api.github.com/user/repos'
      args.Params             ← ⎕NS ''
      args.Params.name        ← 'TestRepository'
      args.Params.description ← 'HttpCommand Example'
      args.Params.auto_init   ← ⊂'true' ⍝ APL's equivalent to JSON true
      ⊢r ← HttpCommand.GetJSON args
[rc: 0 | msg:  | HTTP Status: 401 "Unauthorized" | ⍴Data: ⍬]

Oops! We didn't use our token for authentication and so the request was rejected as unauthorized. We also specify the authentication scheme as 'token', which is what the GitHub API uses.

      args.(AuthType Auth) ← 'token' token
      ⊢r ← HttpCommand.GetJSON args
[rc: 0 | msg:  | HTTP Status: 201 "Created" | ⍴Data: ⍬]

Notice that the shape of the response's payload (Data) is ⍬. This is because we used the GetJSON method and HttpCommand automatically converted the response's JSON payload into a Dyalog JSON object which contains all the information about the newly created repository...

      ]Box on
      r.Data.(name created_at)
┌──────────────┬────────────────────┐
│TestRepository│2022-08-01T21:23:08Z│
└──────────────┴────────────────────┘

Downloading a File

If you want to download a file you can simply use HttpCommand.Get with the URL of the file and then write the response's payload to a local file. Or, you can set the OutFile setting to the local file name and have HttpCommand write the file for you. This is especially advantageous for large files if the server uses "chunked" transfer encoding to send the payload in chunks. In that case, rather than buffering the entire payload in the workspace and then writing to the local file, HttpCommand will write each chunk out as it is received.

      args←⎕NS '' 
      args.OutFile ← '/tmp/test.pdf'
      args.Command ← 'get' 
      args.URL     ← 'https://www.dyalog.com/uploads/files/student_competition/2021_problems_phase1.pdf'
      ⊢r ← HttpCommand.Do args
[rc: 0 | msg:  | HTTP Status: 200 "OK" | 179723 bytes written to c:/tmp/test.pdf]

Twitter Example

The GitHub examples above used the default AuthType of 'token'. Twitter's v2 REST API uses OAuth 2.0's 'bearer' authentication scheme. In this example our super secret bearer token is stored in a variable named bearerToken.

      apiURL ← 'https://api.twitter.com/2/' ⍝ set the base URL for each request
      c ← HttpCommand.New ''
      c.(AuthType Auth) ← 'bearer' bearerToken

First we need to get the id for the Twitter account we want to look at. Twitter returns its response payload in JSON, so we can have HttpCommand automatically convert the JSON into an APL object by setting TranslateData to 1. If there are no errors in the request, Twitter will put the response's data in an object called 'data'.

      c.TranslateData ← 1 ⍝ have HttpCommand translate known data types
      c.URL ← apiURL,'users/by?usernames=dyalogapl'
      ⊢id ← ⊃(c.Run).Data.data.id ⍝ get the id for user "dyalogapl"
36050079
      c.URL ← apiURL,'users/',id,'/tweets?tweet.fields=created_at'
      ⊢ r ← c.Run
[rc: 0 | msg:  | HTTP Status: 200 "OK" | ⍴Data: ⍬]
      ≢ tweets ← r.Data.data ⍝ how many tweets were returned?      
10
      ↑(4↑tweets).((10↑created_at) (50↑text))
 2022-07-26  Remember to submit your solutions to the annual AP 
 2022-07-25  Dyalog ’22 – make sure you register for Dyalog ’22 
 2022-07-20  The APL practice problems site https://t.co/1SEoa5 
 2022-07-15  There are only two weeks until the 10% Early Bird

In a perfect world, all HTTP requests would be well-formed, target hosts would be available, and APIs well-documented. Unfortunately, we don't live in a perfect world and so, in this section, we attempt to provide you with some guidance for troubleshooting requests that aren't working as you expect.

The result namespace returned by HttpCommand.Run contains information that may help in troubleshooting. The display format of the result contains some useful information: * rc is the HttpCommand's return code * msg is a diagnostic message if rc≠0 * HTTP Status indicates the host's HTTP return code and status message * ⍴Data indicates the shape of the payload, if any, received in the response. This is the shape of the payload data after any automatic conversion done by HttpCommand.

First check `rc` and `msg`

If rc is not 0, msg should have relevant information about what went wrong. rc values less than 0 indicate a problem that wasn't Conga-related. rc values greater than 0 are the Conga return code if there was a Conga-related error.

      HttpCommand.Get ''
[rc: ¯1 | msg: No URL specified | HTTP Status:  "" | ⍴Data: 0]

      HttpCommand.Get 'non_existent_url'
[rc: 1106 | msg: Conga client creation failed  ERR_INVALID_HOST  Host 
identification not resolved  | HTTP Status:  "" | ⍴Data: 0]

Check `HttpMessage` and `Data`

If rc=0, but HttpStatus is not what you expect, examine both HttpMessage and Data. In the example below, GitHub returns a URL pointing to their API documentation for getting information about an organization.

      ⊢r ← HttpCommand.Get 'https://api.github.com/orgs/bloofo'
[rc: 0 | msg:  | HTTP Status: 404 "Not Found" | ⍴Data: 109]
      r.HttpMessage
Not Found
      r.Data
{"message":"Not Found","documentation_url":"https://docs.github.com/rest/reference/orgs#get-an-organization"}

Examine the request that `HttpCommand` will send

It can be useful to examine the actual request that HttpCommand will send to the host. Many web services will publish sample requests that you can use for comparison. Check that the headers, particularly content-type, are correct.

The instance setting RequestOnly controls whether HttpCommand will send the request to the host or simply return the request that would be sent to the host. You can set RequestOnly in several ways: * Pass 1 as the optional left argument to the shortcut methods Get, GetJSON, or Do.

       method ← 'post'
       url ← 'www.someurl.net'
       payload ← ⍳10
       headers ← 'a-header' 'some value'
       1 HttpCommand.GetJSON method url payload headers
POST / HTTP/1.1
a-header: some value
Host: www.someurl.net
User-Agent: Dyalog/HttpCommand
Accept: */*
Content-Type: application/json;charset=utf-8
Content-Length: 22

[1,2,3,4,5,6,7,8,9,10]

Pass 1 as the optional left argument to the New shortcut method and then run the Run method. Be sure to set RequestOnly back to 0 before sending the request to the host.

      c ← 1 HttpCommand.New 'get' 'dyalog.com'
      c.Run
GET / HTTP/1.1
Host: dyalog.com
User-Agent: Dyalog/HttpCommand
Accept: */*
      c.RequestOnly←0
      c.Run
[rc: 0 | msg:  | HTTP Status: 200 "OK" | ⍴Data: 20684]

Use the Show instance method.

      c ← HttpCommand.New ''
      c.URL ← 'someurl.com/stuff'
      c.Command ← 'get'
      c.Params ← 'name' 'Drake Mallard'
      'header-name' c.AddHeader 'header value'
      c.Show
GET /stuff?name=Drake%20Mallard HTTP/1.1
header-name: header value
Host: someurl.com
User-Agent: Dyalog/HttpCommand
Accept: */*

Debug Mode

Setting HttpCommand.Debug←1 will disable HttpCommand's error checking and cause HttpCommand to suspend if and where an error occurs.

Setting HttpCommand.Debug←2 will cause HttpCommand to suspend just before it attempts to create the client connection to the host. This will give you the opportunity to examine the request's state and step through the code.

Note: HttpCommand.Debug is a shared setting and will affect all instances of HttpCommand.

Ended: Usage

Reference ↵

Settings ↵

Request-related settings are settings you use to specify attributes of the HTTP request that HttpCommand will process.

Settings

`Command`

Description	The case-insensitive HTTP command (method) for the request. `Command` is not limited to standard HTTP methods like GET, POST, PUT, HEAD, OPTIONS, and DELETE, but can be any string provided that the host has implemented support for it.
Default	`'GET'`
Example(s)	`h.Command←'POST'`

`URL`

Description	The URL for the request. The general format for `URL` is: `[scheme://][userinfo@]host[:port][/path][?query]` At a minimum the `URL` must specify the host.
Default	`''`
Example(s)	`h.URL←'dyalog.com' h.URL←'https://user:pwd@adomain.com:8080/apath/endpoint?name=Drake`
Details	A URL has the general format: `[scheme://][userinfo@]host[:port][/path][?query]` So, a URL can be as simple as just a host name like `'dyalog.com'` or as complex as `'https://username:password@ducky.com:1234/?id=myid&time=1200'` The only mandatory segment is the `host`; `HttpCommand` will infer or use default information when it builds the HTTP request to be sent. scheme - if supplied, it must be either `'http'`, or `'https'` for a secure connection. If not supplied, `HttpCommand` will use `'http'` unless you have specified the default HTTPS port (443) or provided SSL certificate parameters. userinfo - used for HTTP Basic authentication. HTTP Basic authentication has generally been deprecated, but may still be supported by some hosts. If userinfo is supplied, `HttpCommand` will create a proper Base64-encoded `authorization` header. host - the host/domain for the request port - if not supplied, `HttpCommand` will use the default HTTP port (80) unless the HTTPS scheme is used or certificate parameters are specified in which case the default HTTPS port (443) is used. path - the location of the resource within the domain. If not supplied, it's up to the domain's server to determine the default path. query - the query string for the request. If the HTTP method for the request is `'GET'` or `'HEAD'` and request parameters are specified in `Params`, `HttpCommand` will properly format them and append them to the query string. If you choose to supply query string parameters directly, they should be properly URLencoded. (see `URLencode`)

`Params`

Description	The parameters or payload, if any, for the request. `Params` can be a set of name/value pairs or a character vector that is properly formatted for the `ContentType` or `Command` of the request. See details for more information.
Default	`''`
Example(s)	The following examples are equivalent: `h.Params←(('name' 'dyalog') ('age' 39))` `h.Params←'name=dyalog&age=39'` `ns←⎕NS '' ⋄ ns.(name age)←'dyalog' 39 ⋄ h.Params←ns`
Details	The interpretation of `Params` is dependent on `Command` and the content type of the request. The content type is determined by the `ContentType` setting or the presence of a `content-type` header. `Params`, if not empty, is processed as follows: If `Command` is neither `'GET'` nor `'HEAD'` and no content type has been specified, `HttpCommand` will specify the default content type of `'x-www-form-urlencoded'`. If the content type of the request is specified, `HttpCommand` will set the formatted `Params` as the payload of the request. `Params` is formatted based on the content type: `'x-www-form-urlencoded'`: `Params` will be URLEncoded using `UrlEncode`. `'application/json'`: If `Params` is a character vector that is a valid JSON representation,it is left unaltered. Otherwise `Params` will be converted to JSON format using `1 ⎕JSON`. In the case where `Params` is an APL character vector that is also valid JSON, you should JSON encode it prior to setting `Params`. For example, if you have the character vector `'[1,2,3]'` and you want it treated as a string in JSON and not thenumeric array `[1,2,3]`, you should process it yourself using `1 ⎕JSON`. For any other content type, it is the responsibility of theuser to format `Params` appropriately for the content type. If `Command` is either `'GET'` or `'HEAD'` (and no content type has been specified), `HttpCommand` will append the formatted `Params` to the query string of `URL`. In this case, `Params` is formatted as follows: If `Params` is a simple array: If `P←(∊⍕Params)` consists entirely of valid URLEncoded characters, `P` is appended to the query string. (see `ValidFormUrlEncodedChars`) Otherwise `HttpCommand` will URLEncode `P` and append the result to the query string.

`Headers`

Description	The HTTP headers for the request. Specified as one of: a set of name/value pairs. a character vector of of one or more name/value pairs in the form `'name:value'` separated by CRLF (`⎕UCS 13 10`). a vector of character vectors, each in the form `'name:value'`
Default	By default, `HttpCommand` will create the following headers if you have not supplied them yourself: `Host:` the host specified in `URL` `User-Agent: 'Dyalog/HttpCommand 5.0.1'` or whatever the version happens to be `Accept: '/'` If the request content type has been specified, `HttpCommand` will insert an appropriate `content-type` header. If the request has content in the request payload, Conga will automatically supply a `content-length` header.
Example(s)	The following examples are equivalent: `h.Headers←'accept' 'text/html' 'content-type' 'test/plain'` `h.Headers←('accept' 'text/html')('content-type' 'test/plain')` `h.Headers←'accept:text/html',(⎕UCS 13 10),'content-type:text/plain'` `h.Headers←('accept:text/html')('content-type:text/plain')`

`ContentType`

Description	This setting is a convenient shortcut for setting the `content-type` HTTP header of the request. If you happen set both `ContentType` and a `content-type` header, `ContentType` takes precedence.
Default	`''`
Example(s)	`h.ContentType←'application/json; charset=UTF-8'`

`Auth`

Description	This setting is the authentication/authorization string appropriate for the authentication scheme specified in `AuthType`. Used along with AuthType, is a shortcut for setting the authorization HTTP header for requests that require authentication. If `Auth` is non-empty, `HttpCommand` will create an `'authorization'` header and and set its value to `AuthType,' ',Auth`. If you happen set both `Auth` and an `authorization` header, `Auth` takes precedence.
Default	`''`
Example(s)	`h.Auth←'my-secret-token'` `h.Auth←h.Base64Encode 'userid:password' ⍝ HTTP Basic Authentication`
Details	For HTTP Basic Authentication, `Auth` should be set to `HttpCommand.Base64Encode 'userid:password'`. Alternatively, if you provide HTTP Basic credentials in the `URL` as in `'https://username:password@someurl.com'`, `HttpCommand` will automatically generate a proper `'authorization'` header.

`AuthType`

Description	This setting is used in conjunction with Auth and is a character vector indicating the authentication scheme. Three common authentication schemes are: `'basic'` for HTTP Basic Authentication `'bearer'` for OAuth 2.0 token authentication `'token'` for other token-based authentication schemes such as GitHub's Personal Access Token scheme Other values may be used as necessary for your particular HTTP request.
Default	`''`
Example(s)	`h.AuthType←'bearer'` `h.(AuthType Auth)←'basic' (h.Base64Encode 'userid:password')`

`Cookies`

Description	`Cookies` is a vector of namespaces, each containing a representation of an HTTP cookie. It contains cookies sent by the host that may be used in a series of requests to the same host. This setting should be considered read-only.
Default	`''`
Example(s)	`(c ← HttpCommand.New 'get' 'github.com').Run` `[rc: 0 \| msg: \| HTTP Status: 200 "OK" \| ⍴Data: 299959]` `≢ c.Cookies` `3` `↑c.Cookies.(Name Host ('DD-MMM-YYYY'(1200⌶)Creation))` `_gh_sess github.com 05-AUG-2022` `_octo github.com 05-AUG-2022` `logged_in github.com 05-AUG-2022`

Name/Value Pairs

Params, for an appropriate content type, and Headers can be specified as name/value pairs. HttpCommand gives you some flexibility in how you specify name/value pairs. You may use:

A vector of depth 2 or ¯2 with an even number of elements.
('name' 'dyalog' 'age' 39)
A vector of 2-element vectors, HttpCommand will treat each sub-vector as a name/value pair.
(('name' 'dyalog') ('age' 39))
A 2-column matrix where each row represents a name/value pair.
2 2⍴'name' 'dyalog' 'age' 39
A reference to a namespace, HttpCommand will treat the variables and their values in the namespace as name/value pairs.
ns←⎕NS '' ⋄ ns.(name age)←'dyalog' 39
Note that the names will be alphabetical in the formatted output.

In general, you will not need to set any of the Conga-related settings since the HttpCommand's defaults will suffice for almost all requests. For more information on Conga, please refer to the Conga User Guide.

Instance Settings

`BufferSize`

Description	The maximum number of bytes that `HttpCommand` will accept for the HTTP status and headers received from the host.
Default	`200000`
Example(s)	`h.BufferSize←50000 ⍝ set a lower threshold`
Details	By default, when using Conga's HTTP mode, as `HttpCommand` does, `BufferSize` applies only to the data received by the `HTTPHeader` event. The intent is to protect against a maliciously large response from the host. If the data received by the `HTTPHeader` event exceeds `BufferSize`, `HttpCommand` will exit with a return code of 1135.

`WaitTime`

Description	The number of milliseconds that Conga will wait (listen) for a response before timing out.
Default	`5000`
Example(s)	`h.WaitTime←10000 ⍝ wait 10 seconds`
Details	Applies only to the time Conga will wait for a response, not how long `HttpCommand` will wait which is controlled by the `Timeout` setting. In general, this setting should be set to an acceptable amount of time for the host to respond to the HTTP request.

`Cert`

Description	The certificate specification to be used for secure communications over SSL/TLS. If set, `Cert` can be either: an instance of the Conga `X509Cert` class a 2-element vector of character vectors representing the full path names to a public certificate file and a private key file. This is a shortcut that can be used instead of specifying both the `PrivateKeyFile` and `PublicCertFile` settings.
Default	`''`
Example(s)	`h.Init ⍝ initialize the local instance of Conga` `h.Cert← 1⊃h.LDRC.X509Cert.ReadCertUrls ⍝ use the first MS certificate store cert` `h.Cert← 'path-to-certfile.pem' 'path-to-keyfile.pem'`
Details	In most cases when using HTTPS for secure communications, the anonymous certificate that `HttpCommand` will create for you will suffice and you need not set `Cert`. It should only be necessary to assign `Cert` in those cases where you need to specify a certificate for authentication purposes. Note: `Cert` is also a positional argument for the shortcut methods (`Get`, `GetJSON`, `Do`, and `New`) as well as the `HttpCommand` constructor. `Cert` appears after the `Headers` positional argument. For example: `h← HttpCommand.Get 'someurl.com' '' '' ('certfile.pem' 'keyfile.pem')`

`PublicCertFile`

Description	`PublicCertFile` is a character vector containing the full path to a public certificate file when using HTTPS for secure communications. `PublicCertFile` is used in conjunction with `PrivateKeyFile`
Default	`''`
Example(s)	`h.PublicCertFile←'path-to-certfile.pem'`
Details	The use of `Cert` and `PublicCertFile`/`PrivateKeyFile` are mutually exclusive and you should only one or the other. If both `Cert` and `PublicCertFile`/`PrivateKeyFile` are specified, `Cert` takes priority.

`PrivateKeyFile`

Description	`PrivateKeyFile` is a character vector containing the full path to a private key file when using HTTPS for secure communications. `PrivateKeyFile` is used in conjunction with `PublicCertFile`
Default	`''`
Example(s)	`h.PrivateKeyFile←'path-to-keyfile.pem'`
Details	The use of `Cert` and `PublicCertFile`/`PrivateKeyFile` are mutually exclusive and you should only one or the other. If both `Cert` and `PublicCertFile`/`PrivateKeyFile` are specified, `Cert` takes priority.

`SSLFlags`

Description	`SSLFlags` is used when using HTTPS for secure communications. It is a part of the certificate checking process and determines whether to connect to a server that does not have a valid certificate.
Default	`32` which means accept the server certificate without checking it
Example(s)	`h.SSLFlags←8`
Details	For more information on the interpretation of `SSLFlag` values, see Conga User Guide Appendix C. Note: `SSLFlags` is also a positional argument for the shortcut methods (`Get`, `GetJSON`, `Do`, and `New`) as well as the `HttpCommand` constructor. `SSLFlags` appears after the `Cert` positional argument.

`Priority`

Description	The GnuTLS priority string that specifies the TLS session's handshake algorithms.
Default	`'NORMAL:!CTYPE-OPENPGP'`
Example(s)	h.Priority←'SECURE128:-VERS-TLS1.0'
Details	In general, you would use a different value than the default only in rare and very specific circumstances. See GnuTLS Priority Strings" for more information. Note: `Priority` is also a positional argument for the shortcut methods (`Get`, `GetJSON`, `Do`, and `New`) as well as the `HttpCommand` constructor. `Priority` appears after the `SSLFlags` positional argument.

Shared Settings

Shared settings are set in the HttpCommand class and are used by all instances of HttpCommand.

`CongaPath`

Description	The path to Conga resources
Default	`''`
Example(s)	`HttpCommand.CongaPath←'/myapp/dlls/'`
Details	This setting is intended to be used when Conga is not located in the Dyalog installation folder or the current folder, as might be the case when deploying `HttpCommand` as a part of a runtime application. `CongaPath` is used for two purposes: If the `Conga` or `DRC` namespace is not found in the `HttpCommand`'s parent namespace or in the root namespace (`#`), then `CongaPath` should be the path to the folder containing the `conga` workspace from which `HttpCommand` will copy the `Conga` namespace. `CongaPath` should be the path to the folder containing the Conga shared libraries See HttpCommand and Conga and/or Integrating HttpCommand for more information.

`CongaRef`

Description	A reference to the `Conga` or `DRC` namespace or to an initialized instance of the `Conga.LIB` class.
Default	`''`
Example(s)	HttpCommand.CongaRef← #.Utils.Conga
Details	This setting is intended to be used when your application has other components that use Conga. To avoid having multiple copies of Conga, you can set `CongaRef` to point to an existing copy of the Conga API. See HttpCommand and Conga and/or Integrating HttpCommand for more information.

`LDRC`

Description	Once `HttpCommand` has been initialized by processing a request, `LDRC` is a reference to the Conga API that `HttpCommand` is using.
Default	N/A
Example(s)	`HttpCommand.Get 'dyalog.com' ⍝ run a request` `HttpCommand.LDRC.Version` `3 4 1612`
Details	`LDRC` should be treated as a read-only setting. It provides a means to access the Conga API which may be helpful for debugging and informational purposes.

`CongaVersion`

Description	Once `HttpCommand` has been initialized by processing a request, `CongaVersion` is the version number, in major.minor format, of the Conga API that `HttpCommand` is using.
Default	N/A
Example(s)	`HttpCommand.Get 'dyalog.com' ⍝ run a request` `HttpCommand.CongaVersion` `3.4`
Details	`CongaVersion` should be treated as a read-only setting. It provides information which may be helpful for debugging and informational purposes.

Operational settings affect how HttpCommand behaves.

Instance settings

`KeepAlive`

Description	A Boolean which indicates whether `HttpCommand` close the client connection after receiving the response from the host. `1` = keep connection alive `0` = close connection
Default	`1`
Example(s)	`h.KeepAlive←0 ⍝ close the connection`
Details	`KeepAlive` is only applicable when you persist an instance of `HttpCommand` and has no effect when you use the shortcut methods `Get`, `GetJSON`, or `Do` as these methods destroy the `HttpCommand` instance they create when they finish execution. HTTP version 1.1 specifies that connections should be kept alive to improve throughput. Nevertheless, a host may close the connection after sending the response or after a period of inactivity from the client (`HttpCommand`). If the connection is closed, `HttpCommand` will open a new connection on a subsequent request.

`MaxPayloadSize`

Description	The maximum response payload size that `HttpCommand` will accept. `¯1` = no maximum `≥0` = maximum size
Default	`¯1`
Example(s)	`h.MaxPayloadSize←100000 ⍝ set a 100,000 byte limit`
Details	If `MaxPayloadSize` is set to a value `≥0` and the response contains a payload, `HttpCommand` checks the payload size as follows: If the response contains a `content-length` header, its value is compared to `MaxPayloadSize`. If exceeded, `HttpCommand` will return with a return code of `¯5` and no payload. Otherwise, the response payload is sent in chunks and `HttpCommand` will accumulate each chunk and compare the total payload size to `MaxPayloadSize`. If exceeded, `HttpCommand` will return with a return code of `¯5` and whatever payload has been accumulated. In either case, if `MaxPayloadSize` is exceeded, the client connection will be closed.

`MaxRedirections`

Description	The maximum number of redirections that `HttpCommand` will follow. A setting of `¯1` means there is no limit to the number of redirections followed; exposing the possibility of an infinite redirection loop and eventually a `WS FULL` error. A setting of `0` means no redirections will be followed.
Default	`10`
Example(s)	`h.MaxRedirections←0 ⍝ do not follow any redirections`
Details	If the response HTTP status indicates a redirection (3XX) to another URL, `HttpCommand` will retain information about the current request in the `Redirections` element of the result namespace and issue a new request to the new URL. `HttpCommand` will do this up to `MaxRedirections` times.

`Outfile`

Description	`Outfile` has up to 2 elements: `[1]` The name of the file or folder to which `HttpCommand` will write the response payload. If a folder is specified, the file name will be the same as the resource specified in `URL`. `[2]` an optional flag indicating: `0` do not write to an existing, non-empty file (the default) `1` replace the contents, if any, if the file exists `2` append to the file, if it exists
Default	`'' 0`
Example(s)	`c ← HttpCommand.New ''` `c.URL←'https://www.dyalog.com/uploads/files/student_competition/2022_problems_phase2.pdf'` `c.OutFile←'/tmp/'` `c.Run` `[rc: 0 \| msg: \| HTTP Status: 200 "OK" \| 399766 bytes written to c:/tmp/2022_problems_phase2.pdf]`
Details	Output to file is subject to `MaxPayloadSize`.

`RequestOnly`

Description	If set to 1, `HttpCommand` will return the character vector HTTP request that would be sent, instead of actually sending the request.
Default	`0`
Example(s)	`h.RequestOnly←1`
Details	This setting is useful for debugging a request that isn't behaving as you expect. Setting optional left argument of shared methods `Get`, `GetJSON`, `Do`, or `New` to `1` will have the same effect as setting `RequestOnly` as will the instance method `Show`.

`SuppressHeaders`

Description	`SuppressHeaders` is a Boolean setting indicating whether `HttpCommand` should suppress the generation of its default HTTP headers for the request. `1` - suppress the default headers `0` - generate the default headers
Default	`0` which means `HttpCommand` will generate its default headers
Example(s)	`h.SuppressHeaders←1`
Details	`HttpCommand` will only generate headers that you have not specified yourself. `HttpCommand` generates the following default headers: `Host` - the host name in the URL. This header must be supplied. `User-Agent` - `Dyalog-HttpCommand/<version>` `Accept` - `/`. By default, `HttpCommand` will accept any response type `Authorization` - set only if you provide HTTP Basic authentication credentials, or set `Auth` and `AuthType` `Cookie - any saved cookies applicable to the request.`

`Timeout`

Description	The number of seconds that `HttpCommand` will count down to wait for a response. If `Timeout` is positive, `HttpCommand` will wait to receive the complete response from the host. If `Timeout` is negative, the countdown will be reset to `\|Timeout` if Conga has received any data.
Default	`10`
Example(s)	h.Timeout←30
Details	`Timeout` should be set to a time value larger than `WaitTime`. Note that `Timeout` is in seconds and `WaitTime` is in milliseconds.

`TranslateData`

Description	Set this to `1` to have `HttpCommand` attempt to convert the response payload for the following content types. `'application/json'` - `HttpCommand` will use `⎕JSON` `'text/xml'` or `'application/xml' - HttpCommand will use ⎕XML`
Default	`0`
Example(s)	`c←HttpCommand.New 'get' 'https://api.github.com/users/dyalog' c.TranslateData←0 50↑(⎕←c.Run).Data [rc: 0 \| msg: \| HTTP Status: 200 "OK" \| ⍴Data: 1306] {"login":"Dyalog","id":4189495,"node_id":"MDEyOk9y c.TranslateData←1 (⎕←c.Run).Data [rc: 0 \| msg: \| HTTP Status: 200 "OK" \| ⍴Data: ⍬] #.[JSON object]`

Shared Settings

Shared settings are set in the HttpCommand class and are used by all instances of HttpCommand.

`Debug`

Description	Set `Debug` to 1 to turn off `HttpCommand`'s error trapping. In this case if an error occurs, execution of `HttpCommand` will be suspended. Set `Debug` to 2 to have `HttpCommand` stop just before sending the request to the host so that you can examine the request and `HttpCommand`'s current state. Both of these settings can be useful for debugging.
Default	`0`
Example(s)	`c←HttpCommand.New 'get' 'https://api.github.com/users/dyalog'` `c.OutFile←0` `c.Debug←1` `c.Run` `DOMAIN ERROR: Invalid file or directory name` `HttpCmd[130] outFile←∊1 ⎕NPARTS outFile` `∧` `c.Debug←0` `c.Run` `[rc: ¯1 \| msg: DOMAIN ERROR while trying to initialize output file '0' \| HTTP Status: "" \| ⍴Data: 0]`

Ended: Settings

Shared Methods ↵

These "shortcut" methods make it easier to issue HTTP requests or create an instance of HttpCommand.

`Get` - issue an HTTP GET request

Get is probably HttpCommand's most-used feature. It issues an HTTP GET request and returns the response from the server. It can be used for many purposes including interacting with web services, downloading files, and returning the contents of web pages.

Syntax	`r ←{RequestOnly} HttpCommand.Get args`
`args`	Either a vector of positional settings (only `URL` is required) `URL Params Headers Cert SSLFlags Priority` Intermediate positional parameters that are not used should be set to `''` A namespace containing named variables for the settings for the request. `(args ← ⎕NS '').(URL Headers) ← 'someurl.com' ('header1' 'value1')`
`RequestOnly`	(optional) A Boolean indicating: `0` - (default) send the HTTP request and return the response result. `1` - return the formatted HTTP request that HttpCommand would send if `RequestOnly` was `0`.
`r`	If RequestOnly is `0` - a namespace containing the request response. `1` - the formatted HTTP request that `HttpCommand` would send if `RequestOnly` was `0`.
Example(s)	`args ← ⎕NS ''` `args.URL ← 'httpbin.org'` `args.Params ← ('name' 'Donald')('species' 'duck')` `HttpCommand.Get args` `[rc: 0 \| msg: \| HTTP Status: 200 "OK" \| ⍴Data: 339]` `HttpCommand.Get 'httpbin.org/get' (('name' 'Donald')('species' 'duck'))` `[rc: 0 \| msg: \| HTTP Status: 200 "OK" \| ⍴Data: 339]` `1 HttpCommand.Get args` `GET /get?name=Donald&species=duck HTTP/1.1` `Host: httpbin.org` `User-Agent: Dyalog-HttpCommand/5.0.2` `Accept: /`

`GetJSON` - issue a request to a JSON-based web service

GetJSON is used to interact with web services that use JSON for their request and response payloads. It was originally developed as a convenient way to interact with Jarvis, Dyalog's JSON and REST Service framework. Conveniently, it turns out that there are many web services, including GitHub and Twitter, that use JSON as well.

When Command is something other than GET or HEAD, GetJSON will automatically convert APL Params into JSON format. For GET and HEAD, HttpCommand will URLEncode Params in the query string of the URL. The rationale behind this is that GET and HEAD requests should not have content; therefore Params should be included in the query string of URL and it doesn't make a lot of sense to include JSON in the query string. If you really need JSON in the query string, you can use ⎕JSON to convert Params.

GetJSON will attempt to convert any JSON response payload into its equivalent APL representation.

Syntax	`r ←{RequestOnly} HttpCommand.GetJSON args`
`args`	Either a vector of positional settings (`Command` and `URL` are required.) `Command URL Params Headers Cert SSLFlags Priority` Intermediate positional parameters that are not used should be set to `''` A namespace containing named variables for the settings for the request.
`RequestOnly`	(optional) A Boolean indicating: `0` - (default) send the HTTP request and return the response result. `1` - return the formatted HTTP request that HttpCommand would send if `RequestOnly` was `0`.
`r`	If RequestOnly is `0` - a namespace containing the request response. `1` - the formatted HTTP request that `HttpCommand` would send if `RequestOnly` was `0`.
Example(s)	These examples assume you have a `Jarvis` service running at `http://localhost:8080` and a endpoint of `#.sum ← {+/⍵}`. `args ← ⎕NS ''` `args.Command ← 'post'` `args.URL ← 'localhost:8080/sum'` `args.Params ← ⍳1000` `⊢r ← HttpCommand.GetJSON args` `[rc: 0 \| msg: \| HTTP Status: 200 "OK" \| ⍴Data: ⍬]` `r.Data` `500500` `Params ← ('per_page' '3')('page' '1')('sort' 'full_name')` `URL ← 'https://api.github.com/orgs/dyalog/repos'` `⊢r ← HttpCommand.GetJSON 'get' URL Params` `[rc: 0 \| msg: \| HTTP Status: 200 "OK" \| ⍴Data: 3]` `r.Data.full_name` `Dyalog/a3s-Linux Dyalog/APLCourse Dyalog/aplssh`

`Do` - issue a generic HTTP request

Do is essentially the same as Get except that you specify the HTTP method (Command) to use.

Syntax	`r ←{RequestOnly} HttpCommand.Do args`
`args`	Either a vector of positional settings (`Command` and `URL` are required) `Command URL Params Headers Cert SSLFlags Priority` Intermediate positional parameters that are not used should be set to `''` A namespace containing named variables for the settings for the request. `(args ← ⎕NS '').(Command URL Headers) ← 'post' 'someurl.com' ('header1' 'value1')`
`RequestOnly`	(optional) A Boolean indicating: `0` - (default) send the HTTP request and return the response result. `1` - return the formatted HTTP request that HttpCommand would send if `RequestOnly` was `0`.
`r`	If RequestOnly is `0` - a namespace containing the request response. `1` - the formatted HTTP request that `HttpCommand` would send if `RequestOnly` was `0`.
Example(s)	`args ← ⎕NS ''` `args.(Command URL) ← 'post' 'httpbin.org/post'` `args.Params ← ('name' 'Donald')('species' 'duck')` `HttpCommand.Do args` `[rc: 0 \| msg: \| HTTP Status: 200 "OK" \| ⍴Data: 465]`

`New` - create a new instance of `HttpCommand`

New is different than the other shortcut methods in that it returns an instance of HttpCommand. Get, GetJSON, and Do all create an instance, run it, return the response namespace, and upon exit the instance is destroyed.

New can be useful for maintaining state information, like HTTP cookies across multiple requests or keeping the connection to the host open so that subsequent requests to the host are processed without the overhead of re-establishing the connection.

After specifying the settings in the instance, run the Run instance method to execute the request.

Syntax	`instance ←{RequestOnly} HttpCommand.New args`
`args`	Either: a vector, possibly empty, of positional settings `Command URL Params Headers Cert SSLFlags Priority` Intermediate positional parameters that are not used should be set to `''` A namespace containing named variables for the settings for the request. `(args ← ⎕NS '').(Command URL Headers) ← 'post' 'someurl.com' ('header1' 'value1')`
`RequestOnly`	(optional) A Boolean indicating when `Run` is executed: `0` - (default) send the HTTP request and return the response result. `1` - return the formatted HTTP request that HttpCommand would send if `RequestOnly` was `0`.
`instance`	An instance of `HttpCommmand`.
Example(s)	Use a namespace for the request settings: `args ← ⎕NS ''` `args.(Command URL) ← 'post' 'httpbin.org/post'` `args.Params ← ('name' 'Donald')('species' 'duck')` `cmd ← HttpCommand.New args` `cmd.Run` `[rc: 0 \| msg: \| HTTP Status: 200 "OK" \| ⍴Data: 465]` Create an instance of `HttpCommand` and set the settings directly: `cmd ← HttpCommand.New '' ⍝ synonym for cmd ← ⎕NEW HttpCommand` `cmd.(Command URL) ← 'post' 'httpbin.org/post'` `cmd.Params ← ('name' 'Donald')('species' 'duck')` `cmd.Run` `[rc: 0 \| msg: \| HTTP Status: 200 "OK" \| ⍴Data: 465]` Set RequestOnly, display the request that would be sent, then send it `cmd ← 1 HttpCommand.New 'get' 'dyalog.com'` `cmd.Run` `GET / HTTP/1.1` `Host: dyalog.com` `User-Agent: Dyalog-HttpCommand/5.0.4` `Accept: /` `cmd.RequestOnly←0` `cmd.Run` `[rc: 0 \| msg: \| HTTP Status: 200 "OK" \| ⍴Data: 23127]`

Base64 Encoding

Base64 is a binary-to-text encoding scheme that is used to transmit binary data over channels the only reliably support text content. Among Base64's uses are:

Embed images or other binary assets in HTML files.
Encode user credentials for HTTP Basic Authentication

`Base64Decode`

Decode a Base64-encoded string or single-byte integer vector.

Syntax	`out←{cpo} HttpCommand.Base64Decode b64`
`b64`	Either a single-byte integer or character vector to be decoded
`cpo`	(optional) If set to 0 or not supplied, Base64Decode will perform UTF-8 conversion on the result. If set to 1, no conversion will be performed. In almost all cases, you can ignore (and not supply) this argument.
`out`	A character vector representing the decodes base-64 right argument.
Example(s)	`HttpCommand.Base64Decode 'RHlhbG9nIOKNuuKNtOKMig=='` `Dyalog ⍺⍴⌊`

`Base64Encode`

Base64 encode a string or integer vector.

Syntax	`b64←{cpo} HttpCommand.Base64Encode in`
`in`	Either an integer or character vector to be encoded
`cpo`	(optional) If set to 0 or not supplied, Base64Encode will first perform UTF-8 conversion on a character argument. If set to 1, no conversion will be performed. In almost all cases, you can ignore (and not supply) this argument.
`b64`	A character vector representing the base-64 encoding of the right argument.
Example(s)	`HttpCommand.Base64Decode 'RHlhbG9nIOKNuuKNtOKMig=='` `Dyalog ⍺⍴⌊`

URL Encoding

URLs can only be sent over the Internet using the ASCII character set. However, URLs often contain characters outside the ASCII character set. URL encoding converts strings to a format acceptable for transmission over the Internet. URLEncoding is also used to encode payloads for content-type 'application/x-www-form-urlencoded'.

`UrlDecode`

URLDecode a URLEncoded string.

Syntax	`out←HttpCommand.UrlDecode in`
`in`	A URLEncoded string
`out`	The URLDecoding of `in`.
Example(s)	`HttpCommand.UrlDecode 'Dyalog%20%E2%8D%BA%E2%8D%B4%E2%8C%8A'` `Dyalog ⍺⍴⌊` `HttpCommand.UrlDecode 'name=Donald%20Duck'` `name=Donald Duck` `HttpCommand.UrlDecode 'first=Donald&last=O%27Mallard'` `first=Donald&last=O'Mallard`

`UrlEncode`

URLEncode a string or a set of name/value pairs.

Syntax	`out←{name} HttpCommand.UrlEncode in`
`in`	One of: a simple character vector to be URLEncoded a set of name/value pairs
`name`	(optional) The name for the URLEncoded right argument. Applies only in the case where `in` is a simple character vector.
`out`	The URLEncoding of `in`.
Example(s)	`HttpCommand.UrlEncode 'Dyalog ⍺⍴⌊'` `Dyalog%20%E2%8D%BA%E2%8D%B4%E2%8C%8A` `'name' HttpCommand.UrlEncode 'Donald Duck'` `name=Donald%20Duck` `HttpCommand.UrlEncode ('first' 'Donald') ('last' 'O''Mallard')` `first=Donald&last=O%27Mallard`

This section describes HttpCommand shared methods that don't conveniently fit into other categories.

`Version`

Returns HttpCommand version information.

Syntax	`r←HttpCommand.Version`
Result	A 3-element vector of: `'HttpCommand'` The version number The version date
Example	`HttpCommand.Version` `HttpCommand 5.0.2 2022-08-03`

`Documentation`

Returns a link to the documentation for HttpCommand that you can click on to navigate to this documentation.

Syntax	`r←HttpCommand.Documentation`
Example	`HttpCommand.Documentation` `See https://dyalog.github.io/HttpCommand/`

`Upgrade`

Upgrade checks GitHub for a newer version of HttpCommand and if one is found it will be ⎕FIXed in the workspace.

Note: Upgrade will only update the in-workspace copy of HttpCommand and will not update the version of HttpCommand found in the Dyalog installation folder. The installed version of HttpCommand is upgraded when updates to Dyalog APL are installed.

Syntax	`r←HttpCommand.Upgrade`
`r`	a 2-element vector of the return code which is `1` if `HttpCommand` was upgraded to a newer version, `0` if `HttpCommand` on GitHub is not newer than the current version, or `¯1` if there was an error `⎕FIX`ing `HttpCommand` from GitHub. character vector message describing the result
Example	`HttpCommand.Upgrade` `0 Already using the most current version: HttpCommand 5.0.2 2022-08-03`

Ended: Shared Methods

The methods documented in this section are instance methods and may be called from an instance of HttpCommand. In the documentation below, we will use the name instance to represent an instance of HttpCommand.

`Run`

Run "runs" the HTTP request defined by the instance's settings. Run is called internally by the shared "shortcut" methods Get, GetJSON, and Do.

Syntax	`result←instance.Run`
`result`	`result` depends on the `RequestOnly` setting. If `RequestOnly=1`, `Run` will return, if possible, the HTTP request that would be sent if `RequestOnly` was set to `0`. If `HttpCommand` cannot form a proper HTTP request, `Run` will return a result namespace with pertinent information. If `RequestOnly=0`, `Run` will return a result namespace containing the result of attempting to build and send the HTTP request specified by the instance's settings.
Examples	`instance←HttpCommand.New ''` `instance.RequestOnly←1` There isn't sufficient information for `HttpCommand` to build a proper HTTP request. `⊢result ← instance.Run` `[rc: ¯1 \| msg: No URL specified \| HTTP Status: "" \| ⍴Data: 0]` `instance.URL←'dyalog.com'` Now, there's enough information to form a proper HTTP request. `⊢result ← instance.Run` `GET / HTTP/1.1 Host: dyalog.com User-Agent: Dyalog-HttpCommand/5.0.3 Accept: /` `instance.RequestOnly←0` `⊢result ← instance.Run` `[rc: 0 \| msg: \| HTTP Status: 200 "OK" \| ⍴Data: 23127]`

`Show`

Returns the HTTP request that HttpCommand would send when Run is executed and RequestOnly is 0. Show is equivalent to setting RequestOnly to 1 and running Run.

Syntax	`r←instance.Show`
Example	`instance ← HttpCommand.New 'get' 'dyalog.com'` `instance.Show` `GET / HTTP/1.1` `Host: dyalog.com` `User-Agent: Dyalog-HttpCommand/5.0.3` `Accept: /`

`Config`

Config returns the current state of all HttpCommand settings.

Syntax	`r←instance.Config`
`r`	A 2-column matrix where `[;1]` contains the setting names `[;2]` contains the corresponding setting values
Example	`instance←HttpCommand.New 'get' 'dyalog.com'` `instance.Config` `Auth` `AuthType` `BufferSize 200000` `Cert` `Command get` `CongaPath` `CongaRef` `CongaVersion` `ContentType` `Cookies` `Debug 0` `Headers` `KeepAlive 1` `LDRC not set` `MaxPayloadSize ¯1` `MaxRedirections 10` `OutFile` `Params` `Priority NORMAL:!CTYPE-OPENPGP` `PrivateKeyFile` `PublicCertFile` `RequestOnly 0` `SSLFlags 32` `SuppressHeaders 0` `Timeout 10` `TranslateData 0` `URL dyalog.com` `WaitTime 5000`

`Init`

Init initializes Conga, Dyalog's TCP/IP utility library. Normally, HttpCommand will initialize Conga when Run is first called. Init is intended to be used when the user wants to "tweak" Conga prior to Run being executed. It's very unlikely that you'll ever need to use Init.

Syntax	`r←instance.Init`
`r`	a 2-element vector of `[1]` the return code. `0` means Conga is initialized. Non-`0` indicates some error occurred in Conga initialization. `[2]` a message describing what went wrong if the return code is not `0`
Example	`instance←HttpCommand.New ''` `instance.CongaPath←'/doesnotexist'` `instance.Init` `¯1 CongaPath "c:/doesnotexist/" does not exist`

There are two sets of headers associated with an HTTP request - the request headers and the response headers. The methods described here deal with request headers.

HttpCommand's request headers are stored in the Headers setting which is a 2-column matrix of name/value pairs. Header names are case-insensitive; header values are not. While you can manipulate the Headers array directly, HttpCommand has three methods to manage Headers that accommodate the case-insensitive nature of header names.

Note: The examples below were run using ]boxing on.

`AddHeader`

AddHeader will add a header if a header with that name does not already exist. Use SetHeader to set a header regardless if one with the same name already exists.

Syntax	`name instance.AddHeader value`
`value`	the value for the header
`name`	the name for the header
Example	`instance←HttpCommand.New ''` `'My-Header' instance.AddHeader 'Drake Mallard'` `instance.Headers` `┌─────────┬─────────────┐` `│My-Header│Drake Mallard│` `└─────────┴─────────────┘` `AddHeader` will not replace an existing header with the same case-insensitive name `'my-header' instance.AddHeader 'Daffy Duck'` `instance.Headers` `┌─────────┬─────────────┐` `│My-Header│Drake Mallard│` `└─────────┴─────────────┘`

`SetHeader`

SetHeader will set a header, replacing one of the same name if it already exists.

Syntax	`name instance.SetHeader value`
`value`	the value for the header
`name`	the name for the header
Example	`instance←HttpCommand.New ''` `'My-Header' instance.SetHeader 'Drake Mallard'` `instance.Headers` `┌─────────┬─────────────┐` `│My-Header│Drake Mallard│` `└─────────┴─────────────┘` `SetHeader` will replace an existing header with the same case-insensitive name `'my-header' instance.SetHeader 'Daffy Duck'` `instance.Headers` `┌─────────┬──────────┐` `│My-Header│Daffy Duck│` `└─────────┴──────────┘`

`RemoveHeader`

RemoveHeader removes a header. If the header does not exist, RemoveHeader has no effect.

Syntax	`instance.RemoveHeader name`
`name`	the case-insensitive name of the header to remove
Example	`'My-Header' instance.SetHeader 'Daffy Duck'` `'another' instance.SetHeader 'some value'` `instance.Headers` `┌─────────┬──────────┐` `│My-Header│Daffy Duck│` `├─────────┼──────────┤` `│another │some value│` `└─────────┴──────────┘` `instance.RemoveHeader 'my-header'` `instance.Headers` `┌───────┬──────────┐` `│another│some value│` `└───────┴──────────┘`

Result Namespace ↵

For the purposes of this document, we will refer to result as the result namespace returned by Run when RequestOnly=0. result contains information about the request and the response, if any, from the host. Some settings from the request are copied into result.

`rc`

The operational return code. This indicates if HttpCommand was able to form and send an HTTP request and process the response from the host. The sign of rc indicates:

rc=0 no operational error occurred.
rc<0 an operational error occurred.
rc>0 a Conga error occurred.

rc is further described in Messages and Return Codes. To verify that an HTTP request was successful, first check that result.rc=0 and then that result.HttpStatus is a HTTP status code you would expect from the host.

`msg`

The operational status message. If rc=0, msg will generally be empty (''). msg will contain (hopefully) helpful information as to the nature of the error or condition. msg is further described in Messages and Return Codes.

`Elapsed`

The number of millseconds that the request took to process.

`OutFile`

OutFile is copied from the OutFile setting so that, if the response payload is written to file, the result namespace has a record of the file name.

`BytesWritten`

If OutFile is non-empty, and the HTTP request wassuccessful, BytesWritten is the number of bytes written to that file named by OutFile. Otherwise, BytesWritten is set to ¯1.

For the purposes of this document, we will refer to result as the result namespace returned by Run when RequestOnly=0. result contains information about the request and the response, if any, from the host.

Request-related elements in the result namespace are either copied or derived from certain request settings. Not all request settings are included in the result namespace, but they are retained in the HttpCommand instance.

`Command`

The case-insensitive HTTP command (method) for the request.

`Host`

The final host to which the HTTP request was sent. This may be different from the host specified in the URL setting if the request was redirected.

`Path`

The final path for the requested resource. This may be different from the resource specified in the URL setting if the request was redirected.

`Port`

The port that the request was sent to. If the request was redirected, this may be different from the port specified in the URL setting or the original default port if no port was specified in URL.

`Secure`

A Boolean indicating whether the final request was sent using HTTPS. If the request was redirected, this may be reflect a different secure state from the original request.

`URL`

The final URL of the HTTP request. If the request was redirected, this may be different from the URL setting.

or the purposes of this document, we will refer to result as the result namespace returned by Run when RequestOnly=0. result contains information about the request and the response, if any, from the host. Some settings from the request are copied into result.

`HttpStatus`

The integer HTTP response status code returned by the host. Status codes are split into specific ranges:

1xx: Informational - Request received, continuing process
2xx: Success - The action was successfully received, understood, and accepted
3xx: Redirection - Further action must be taken in order to complete the request
4xx: Client Error - The request contains bad syntax or cannot be fulfilled
5xx: Server Error - The server failed to fulfill an apparently valid request

`HttpMessage`

A character vector indicating the HTTP message coresponding to the HTTP status code.

`HttpVersion`

A character vector indicating the HTTP version used by the host.

`Data`

In most cases Data will be the response's character vector payload. Data is not populated if the OutFile setting is set - the payload is instead written, as received, to file.

The Content-Type response header will indicate the type of data in the payload and the Content-Encoding header, if present, will indicate the compression scheme used if the payload is compressed.

If the response payload is compressed using 'deflate' or 'gzip' encoding HttpCommand will uncompress the payload.

If the Content-Type header indicates that the UTF-8 character set is used, HttpCommand will convert the payload to UTF-8.

HttpCommand can translate certain content types into a format that may be more useful in the APL environment. If the TranslateData is set to 1, HttpCommand will attempt to translate application/json content using ⎕JSON, and text/xml and application/xml using ⎕XML.

`Headers`

Headers is the 2-column matrix of response header name/value pairs.

Note: There are two sets of headers in an HTTP request/response transaction - the request headers that are sent to the host, and the response headers that are sent back from the host.

`∇GetHeader`

GetHeader is a utility function included in the result namespace that makes it easier to extract response header values. GetHeader accommodates the case-insensitive nature of header names. If the header does not exist, GetHeader returns ''.

Syntax	`value ← {headers} result.GetHeader name`
`name`	A character vector header name, or vector of header names
`headers`	The optional header array to extract from. By default, `GetHeader` will use the response's `Headers` array. You may also pass the `Headers` element of a redirection (see `Redirections`).
`value`	The header value(s) corresponding to the header names specified in `name`, or the empty vector `''` for header name(s) that don't exist.
Example(s)	`result.GetHeader 'content-type'` `text/html; charset=utf-8` `⍝ Note: there is no 'fred' header` `result.GetHeader 'content-type' 'fred' 'server'` `┌────────────────────────┬┬──────────────────────┐` `│text/html; charset=utf-8││Apache/2.4.18 (Ubuntu)│` `└────────────────────────┴┴──────────────────────┘`

`Cookies`

Cookies is a vector of namespaces, one per cookie set by the host. Cookies are frequently used to maintain a state between the client (HttpCommand) and the host. When using an instance of HttpCommand, the instance will retain a copy of Cookies and apply appropriate cookies to subsequent requests.

The sample below shows the names and first 50 characters of the values of 3 cookies returned by GitHub.com. The first cookie holds the session ID.

      'Name' 'Value'⍪↑(HttpCommand.Get'github.com').Cookies.(Name((50∘⌊∘≢↑⊢)Value))
 Name       Value                                              
 _gh_sess   A0ZezbnUgtkiq1lBubH7mrXclGhvhcCFKRUbAb045wGNT2Hlma 
 _octo      GH1.1.809596667.1660313286                         
 logged_in  no

`PeerCert`

If the connection to the host uses HTTPS as its protocol which means it is a secure connection, then PeerCert contains the X509 certificate of the host.

      ⊃(HttpCommand.Get 'dyalog.com').PeerCert.Formatted.(Issuer Subject)
┌──────────────────────────┬───────────────┐
│C=US,O=Let's Encrypt,CN=R3│CN=*.dyalog.com│
└──────────────────────────┴───────────────┘

`Redirections`

Redirections is a vector of namespaces, one per HTTP redirect response from the host. An HTTP redirect is a special kind of response from the host that directs the client to another location for the resource that's being requested. When a web browser like Google Chrome or Mozilla Firefox receives a redirect response, they automatically reissue the request for the resource at the new location. The new location is specified in the response's Location header. A redirection can itself be redirected and can even result in an infinite redirection loop. HttpCommand will also follow the redirections, but it also retains information about each redirection in Redirections. The MaxRedirections setting can limit the number of redirections that HttpCommand will follow.

Each namespace in Redirections has the following elements:

`HttpStatus`	The HTTP status for the request.
`HttpMessage`	The corresponding HTTP status message for the request.
`HttpVersion`	The HTTP version that was used in sending the response.
`Headers`	The headers that were sent by the host. There should be a `'Location'` header that indicates the redirection location for the requested resource
`URL`	The URL that was used and produced this redirection.

The example below uses httpbin.org/relative_redirect/2 to redirect the request twice.

      ⊢result ← HttpCommand.Get 'http://httpbin.org/relative_redirect/2'
[rc: 0 | msg:  | HTTP Status: 200 "OK" | ⍴Data: 266]

Examining the information in the 2 redirections shows that our original URL, 'httpbin.org/relative-redirect/2', was redirected to /relative-redirect/1 which was subsequently redirected to /get.

      'URL' 'Was redirected to'⍪result.(Redirections.URL,⍪Redirections.Headers GetHeader¨⊂'location')
┌───────────────────────────────┬────────────────────┐
│URL                            │Was redirected to   │
├───────────────────────────────┼────────────────────┤
│httpbin.org/relative-redirect/2│/relative-redirect/1│
├───────────────────────────────┼────────────────────┤
│httpbin.org/relative-redirect/1│/get                │
└───────────────────────────────┴────────────────────┘

Ended: Result Namespace

`rc` Values

The value of rc may be interpreted as follows:

Error Type	`rc` Value	Description
No error	`0`	`HttpCommand` was able to format and send the request and receive a response without error.
Conga error	`>0`	If `rc` is a positive integer, it is a return code that is signalled by Conga when sending the request and receiving the response. If Conga detects an error at the operating system level, it will return the operating system return code. For instance, 10013 is the Microsoft Windows return code for "port in use".
Translation Error	`¯2`	You specified `TranslateData←1` and `HttpCommand` was unable to translate an XML or JSON response payload using `⎕XML` or `⎕JSON` respectively or you used `GetJSON` and response content-type header was not `'application/json'`.
Everything else	`¯1`	This is the generic return code for any error condition that is not signalled by Conga.

In general if rc=0 then msg will be empty. There are circumstances where msg will contain an informational message even if rc=0. These messages are included below.

In the following message descriptions, items that are enclosed in {} represent the value of that named item. For instance, {BufferSize} represents the value of the BufferSize setting. In many cases, the message text itself is sufficient to explain the condition and its resolution, but we document them nonetheless for completeness.

"Catch All" Messages

Message	`Unexpected {ERROR} at {location}`
Description	You should rarely, if ever, see this message. It is returned when an error occurred that wasn't anticipated or otherwise handled.
Resolution	Try to find a consistently reproduceable example and contact Dyalog support.

Message	`Invalid HttpCommand setting(s): {setting name(s)}`
Description	You passed a namespace argument `New`, `Get`, `GetJSON`, or `Do` or as a constructor argument to `⎕NEW` and that namespace contained named elements that are not valid `HttpCommand` setting names.
Resolution	Remove the named elements that are not `HttpCommand` setting names.

Message	`Stopped for debugging... (Press Ctrl-Enter)`
Description	This message is not returned in the result namespace, but is displayed when you have set `Debug←2` and `HttpCommand` has stopped intentionally just prior to establishing the connection to the host.
Resolution	Pressing Ctrl-Enter will open the debugger and trace into `HttpCommand`. Entering `→⎕LC+1 will resume execution.`

Non-Conga Setting Messages

These messages report problems with settings that prevent HttpCommand from creating a proper HTTP request to send to the host.

Message	`No URL specified`
Description	You didn't specify a `URL` setting.
Resolution	Specify a valid `URL`.

Message	`URL is not a simple character vector`
Description	The `URL` setting isn't a simple character vector.
Resolution	Make sure `URL` isn't nested or non-character.

Message	`Invalid host/port: {host}`
Description	You specified a URL that has a hostname followed by a colon (`':'`) followed by something that does not parse as an integer. For example: `'dyalog.com:ducky'`
Resolution	Correct (or omit) the bad port specification.

Message	`No host specified`
Description	`HttpCommand` couldn't find a host in the `URL` setting.
Resolution	Ensure your `URL` conforms to the specification.

Message	`Invalid port: {port}`
Description	The port you specified is not in the range 1-63535.
Resolution	Specify an integer port number in the range 1-65535.

Message	`Invalid protocol: {protocol}`
Description	`HttpCommand` thinks you've specified a protocol other than `'http://'` or `'https://`.
Resolution	`HttpCommand` supports only the HTTP and HTTPS protocols. If you have not specified the protocol in your `URL` setting, `HttpCommand` will default to use HTTP. HTTPS will be used if you've specified either the default secure port (443) or supplied certificate information.

Message	`Cookies are not character`
Description	`Cookies` are specified as name/value pairs. All of the names and values need to be character.
Resolution	Make sure `Cookies` names and values are character.

Message	`Headers are not character`
Description	`Headers` may be specified as name/value pairs. All of the names and values need to be character.
Resolution	Make sure `Headers` names and values are character.

Message	`Improper header format`
Description	You used a value for `Headers` that did not conform to one of the formats described in the `Headers` specification.
Resolution	Ensure that `Headers` conforms to the specification

Message	`Output file folder {foldername} does not exist.`
Description	The folder in your {OutFile} setting does not exist. `HttpCommand` will create the output file, if necessary, in an existing folder but `HttpCommand` will not create a new folder.
Resolution	Either create the folder or change the `OutFile` setting.

Message	`No filename specified in OutFile or URL`
Description	If you do not specify a file name in `OutFile`, `HttpCommand` will attempt to use the file name specified in `URL`. However, if there is also no file name in `URL`, you will receive this message.
Resolution	Specify a file name in either `OutFile` or `URL`.

Message	`Output file "{OutFile}" is not empty`
Description	You specified the name of an existing, non-empty, file in `OutFile` but did not specify to overwrite or append to the file.
Resolution	Either specify a filename of a file that doesn't exist or is empty, or specify that the file is to be overwritten or appended to.

Conga Setting Messages

The messages in this section are related to the Conga settings in HttpCommand, not the actual execution of Conga code whose messages are described in "Conga Execution"-Related Messages.

Message	`CongaRef {CongaRef} does not point to a valid instance of Conga`
Description	You specified a `CongaRef` setting that is not the name of, or a reference to, the `Conga` or the `DRC`namespace.
Resolution	Check that `CongaRef` is actually the name of, or a reference to, the `Conga` or the `DRC`namespace.

Message	`{location}.{namespace} does not point to a valid instance of Conga`
Description	`HttpCommand` searches for the Conga API according to these rules and found a namespace named either `Conga` or `DRC`, but that namespace is not a valid Conga namespace.
Resolution	Either set `CongaRef` to point to a proper `Conga` or `DRC` namespace, or remove/rename the offending namespace.

Message	`CongaPath "{CongaPath}" does not exist`
Description	You specified a `CongaPath` that could not be found.
Resolution	Set `CongaPath` to the name of an existing folder containing the Conga shared libraries, or do not set `CongaPath` and let `HttpCommand` use the Dyalog installation folder.

Message	`CongaPath "{CongaPath}" is not a folder`
Description	You specified a `CongaPath` that exists, but is not a folder.
Resolution	Set `CongaPath` to the name of an existing folder containing the Conga shared libraries, or do not set `CongaPath` and let `HttpCommand` use the Dyalog installation folder.

Message	`{Conga API} was copied from {path}, but is not valid`
Description	`HttpCommand` was able to copy either the `Conga` or the `DRC` namespace according to the procedure described in Conga Default Behavior, but the namespace was not a valid Conga API.
Resolution	Ensure that conga workspace from which `HttpCommand` attempted to copy the Conga API is proper and the same version as the shared libraries. See Integrating `HttpCommand`.

Message	`Neither Conga nor DRC were successfully copied`
Description	`HttpCommand` was unable to copy either the `Conga` or the `DRC` namespace according to the procedure described in Conga Default Behavior.
Resolution	The most likely cause is that there is no conga workspace in the paths from which `HttpCommand` tried to copy. See Integrating `HttpCommand`.

Message	`Not found PublicCertFile "{PublicCertFile}"` `Not found PrivateKeyFile "{PrivateKeyFile}"` `Not found PublicCertFile "{PublicCertFile}" and PrivateKeyFile "{PrivateKeyFile}"`
Description	One or both of the file names you specified for `PublicCertFile` and `PrivateKeyFile` does not exist.
Resolution	Set `PublicCertFile` and `PrivateKeyFile` to the names of certificate files that actually exist.

Message	`PublicCertFile is empty` `PrivateKeyFile is empty`
Description	You have specified one of `PublicCertFile` or `PrivateKeyFile`, but not the other.
Resolution	Be sure to set both `PublicCertFile` and `PrivateKeyFile`.

Message	`Invalid certificate parameter`
Description	You specified a value for `Cert` that is not a valid instance of Conga's `X509` class.
Resolution	Make sure that `Cert is a valid instance of the X509 class.`

Message	`Unable to decode PublicCertFile "{PublicCertFile}" as certificate`
Description	An error occurred when Conga attempted to use `X509Cert.ReadCertFromFile` with the value you specified for `PublicCertFile`.
Resolution	Make sure `PublicCertFile` contains a valid certificate. See the Conga User Guide for more information.

Conga Execution Messages

The messages in this section are returned when Conga returns a non-zero return code, or an unexpected Conga event occurs. If there is a non-zero return code from Conga, it is returned in rc.

HttpCommand uses Conga's "HTTP" mode, where Conga will parse the response from the host. Some of the messages occur when the host sends an ill-formed response and Conga cannot parse it as HTTP. The ill-formed data is returned in the Data element of HttpCommand's result namespace.

Message	`Could not initialize Conga due to...`
Description	Conga could not be initialized for one of many reasons. The remainder of the message should give some indication as to what the problem was. Examples of problems include: the `Conga` or `DRC` namespace that `HttpCommand` found is not a valid Conga interface. the `CongaPath` setting does not point to an existing folder containing the Conga shared libraries. `HttpCommand` was unable to find a valid `Conga` or `DRC` namespace and could not copy either from the `conga` workspace.
Resolution	The resolution will depend on the circumstances described in the message. The Conga User Guide and/or the Integrating `HttpCommand`section could be helpful here.

Message	`Conga client creation failed...`
Description	This will occur if Conga is not able to create a client connection to the host. The remainder of the message will indicate the specific Conga error message.
Resolution	The resolution will depend on the circumstances described in the message. See the Conga User Guide.

Message	`Conga could not parse the HTTP response`
Description	The response from the host is not a properly formatted HTTP message.
Resolution	Inspect the data returned by the host in the `Data` element of the result.

Message	`Response payload not completely received`
Description	This occurs when `HttpCommand` was able to receive and parse the HTTP headers from the host but, during the receipt of the response payload, an error occurred.
Resolution	Attempt the request again. If the error persists, examine the headers and payload data, if any, returned by the host.

Message	`Conga error processing your request: {Conga return code}`
Description	Conga encountered some error that wasn't otherwise diagnosed or trapped.
Resolution	See the Conga User Guide for guidance.

Message	`Socket closed by server`
Description	During processing of the request, the server unexpectedly closed the connection to `HttpCommand`.
Resolution	Check that your request is proper for the host. See the Troubleshooting guide.

Message	`Unhandled Conga event type: {Conga event}`
Description	This should never occur, but for the sake of paranoia, we coded for the possibility of a Conga event that we didn't anticipate.
Resolution	Try to find a consistently reproduceable example and contact Dyalog support.

Message	`Conga wait error: {Conga return code}`
Description	An error occurred while Conga was waiting for a response from the host.
Resolution	Re-attempt the request. Examine any data or headers that may have been received for clues.

Message	`Could not set DecodeBuffers on Conga client "{Conga client name}": {Conga return code}`
Description	This is another error that should never occur.
Resolution	Try to find a consistently reproduceable example and contact Dyalog support.

Message	`Conga failed to parse the response HTTP header`
Description	Conga received an HTTPHeader event but was unable to parse the received data as HTTP headers.
Resolution	The unparseable data is returned in the `Data` element of the result namespace and may provide insight as to the problem.

Message	`Conga failed to parse the response HTTP chunk`
Description	Conga received an HTTPChunk event but was unable to parse the received data as an HTTP chunk.
Resolution	The unparseable data is returned in the `Data` element of the result namespace and may provide insight as to the problem.

Message	`Conga failed to parse the response HTTP trailer`
Description	Conga received an HTTPTrailer event but was unable to parse the received data as an HTTP trailer.
Resolution	The unparseable data is returned in the `Data` element of the result namespace and may provide insight as to the problem.

Message	`Conga error while attempting to send request: {Conga return code}`
Description	Conga was able to establish a connection to the host, but encountered an error when attempting to send the HTTP request.
Resolution	Examine the request being sent as described in the Troubleshooting guide. The Conga User Guide may also be helpful.

Message	`Unable to set EventMode on Conga root`
Description	This is another message that should never occur.
Resolution	Try to find a consistently reproduceable example and contact Dyalog support.

Operational Messages

Message	`{ERROR} while trying to initialize output file "{OutFile}"`
Description	You specified an `OutFile` setting and an error occured while trying to initialize the file.
Resolution	{ERROR} should provide information that will help determine the resolution.

Message	`Response header size exceeds BufferSize {BufferSize}`
Description	The size of the response's HTTPHeader message exceeded the value set for `BufferSize.`
Resolution	`BufferSize` is intended to help counteract maliciously large message headers. If the response headers are expected to be very large, increase the value of `BufferSize`.

Message	`Payload length exceeds MaxPayloadSize`
Description	The `content-length` header in the response had a value that exceeded the `MaxPayload` setting.
Resolution	`MaxPayloadSize` is intended to help counteract maliciously large messages. If the response payload is expected to be very large, increase the value of `MaxPayloadSize`

Message	`Request timed out before server responded`
Description	The host did not respond within the number of seconds specified in the `Timeout` setting.
Resolution	If you expect the host to be slow in responding, increase the magnitude of the `Timeout` setting.

Message	`Redirection detected, but no "location" header supplied.`
Description	The host responded with a 3XX HTTP status (redirection), but the response did not contain a `location` header indicating where to redirect the request.
Resolution	This is a server-side problem.

Message	`Too many redirections ({MaxRedirections})`
Description	The host sent more 3XX HTTP status responses (redirections) than `MaxRedirections` specified.
Resolution	`MaxRedirections` is intended to prevent redirection loops or too many redirection hops. If the redirections are legitimate, increase the value of `MaxRedirections`.

Message	`Could not translate XML payload`
Description	If you specify `TranslateData←1` and the response content-type header contains `'text/xml'` or `'application/xml'`, `HttpCommand` will attempt to use `⎕XML` to translate the payload. If `⎕XML` fails, this message is returned and `rc` is set to `¯2`.
Resolution	This is probably due to the response payload containing incorrectly formatted XML. The untranslated payload is returned in `Data`.

Message	`Could not translate JSON payload`
Description	If you specify `TranslateData←1` or use the `GetJSON` shared method and the response content-type header contains `'application/json'`, `HttpCommand` will attempt to use `⎕JSON` to translate the payload. If `⎕JSON` fails, this message is returned and `rc` is set to `¯2`.
Resolution	This is probably due to the response payload containing incorrectly formatted JSON. The untranslated payload is returned in `Data`.

Message	`Response content-type is not application/json`
Description	You used `GetJSON` and the response content-type header is not `'application/json'`. `rc` is set to `¯2` and `Data` contains the response payload.
Resolution	`GetJSON` expects the response content-type to be `'application/json'`. If the host is expected to return some other content-type, consider using a method other than `GetJSON`.

Informational Messages

These messages are issued to inform you that, while there was no actual error,

Message
Description
Resolution

Message	`Connection properties changed, connection reset`
Description	Unless otherwise specified, an instance of `HttpCommand` will attempt to keep the connection for the last request open. If any of the connection properties change on a subsequent request, the old connection closed and a new connection is established.
Resolution	No resolution necessary.

Message	`Unhandled content-encoding: {content-encoding}`
Description	This message occurs if the response's content-encoding header is something other than `'gzip'` or `'deflate'`.It is intended to inform you that `HttpCommand` did not automatically decode the payload.
Resolution	Your application will need to decode the payload.

Other Messages

These messages are not found in the msg element of result namespace, but are issued when using other features of HttpCommand.

Message	`Could not ⎕FIX file: {ERROR}{: Message}`
Description	The `Fix` method was not able to `⎕FIX` the file content returned as the response payload.
Resolution	Ensure that the requested file contains `⎕FIX`-able contents.

Message	`Could not ⎕FIX new HttpCommand: {ERROR}{: Message}`
Description	The `Upgrade` method was not able to `⎕FIX` a newer `HttpCommand`.
Resolution	See `Upgrade`. If the problem persists, contact Dyalog support.

Message	`Upgraded to {new-version} from {old-version}`
Description	The `Upgrade` method was able to `⎕FIX` a newer version of `HttpCommand`.
Resolution	See `Upgrade`.

Message	`Already using the most current version {version}`
Description	The `Upgrade` method did not find a newer version of `HttpCommand`.
Resolution	See `Upgrade`.

Ended: Reference

Advanced Topics ↵

HttpCommand uses Conga, Dyalog's TCP/IP utility library, for communications. HttpCommand requires Conga version 3.0 or later. Conga consists of two elements:

Two shared library files whose names begin with "conga" and are found in the Dyalog installation folder. The names of the files varies based on the platform they are running on and the specific version of Conga; for instance conga34_64.dll and conga34ssl64.dll are the shared library files for Conga version 3.4 for the 64-bit version of Dyalog for Windows.
An APL-based API to communicate with the shared libraries. There are two versions of the API both of which are available in the conga workspace.
- A namespace named Conga which was introduced with Conga version 3.0 and implements behavior that makes it easier to run multiple Conga-based utilities in the same workspace. Conga is the recommended API version to use.
- A namespace named DRC which is retained for backward-compatibility with applications which use earlier versions of Conga. DRC should only be used in an application where backward compatibility is necessary.

The Conga API looks for a matching version of the shared libraries; as such version of the API and the shared libraries must be the same. For more information on Conga please refer to the Conga User Guide.

Default Behavior

When first run, HttpCommand will attempt find or copy the Conga API and then initialize it. HttpCommand will attempt to use the Conga version of the API in preference to the DRC version. In general, all of this is transparent to the user.

Look in the current workspace for the Conga API:
- First look in the namespace where HttpCommand resides for a namespace named either Conga or DRC, in that order.
- Failing that, look in the root namespace # for a namespace named either Conga or DRC, in that order.
If neither version of the API is found in the current workspace, HttpCommand will attempt to copy the API (first Conga then DRC) from the conga workspace. The API is copied into the HttpCommand class which means there will be no additional footprint in workspace. HttpCommand will attempt to copy the API as follows:
- If the DYALOG environment variable exists, use its folder. Otherwise use the folder from the command line used to start Dyalog.
- If that fails, then attempt to copy from the "current" folder as determined by ⊃1 ⎕NPARTS ''
If the API was found or successfully copied, HttpCommand will initialize Conga as follows:
- If the Conga version of the API is used, HttpCommand will initialize it with a root name of 'HttpCommand'.
- If the DRC version of the API is used, HttpCommand will simply initialize it. As DRC does not support multiple roots, care should be taken if other Conga-using utilities also reside in the workspace.
If the API was successfully initialized, a reference to the API root can be found in HttpCommand.LDRC.

Overriding Default Locations

There are two methods to tell HttpCommand's default behavior, both of which involve setting a shared public field in the HttpCommand class.

If you have the Conga namespace in your workspace in other than default locations the HttpCommand will search, HttpCommand.CongaRef can be used to specify its location. CongaRef can be an actual reference to the namespace or a character array representing the location. For instance:
HttpCommand.CongaRef←#.Utils.Conga or
HttpCommand.CongaRef←'#.Utils.Conga'
This can be useful when integrating HttpCommand into an application that also uses Conga.
HttpCommand.CongaPath can be used to specify the path to the shared library files and optionally the conga workspace. This can be useful when bundling HttpCommand in a distributed application. For instance:
HttpCommand.CongaPath←(⊃1 ⎕NPARTS ''),'/conga/' would tell HttpCommand to find the shared libraries in the /conga/ subfolder of the current folder. See Integrating HttpCommand for more information.

Using Other Versions of Conga

If you need to use a version of Conga other than the one in the Dyalog installation folder, there are two ways to accomplish this:

Put the shared libraries and the conga workspace in a folder and set CongaPath to point to that folder.
Put the Conga namespace in your workspace (pointing CongaRef to it if necesssary) and the shared libraries in a folder set CongaPath to point to that folder.

HTTPS

In the HTTP world, secure communications is accomplished by using the HTTPS protocol which uses Transport Layer Security (TLS) or Secure Sockets Layer (SSL) to encrypt communications. Both the client (HttpCommand) and the server use certificates for secure communications. The certificates serve a couple of purposes:

They provide bi-directional encryption of the data sent across the link.
They can help provide authentication of the client and/or the server.

`HttpCommand` Behavior

HttpCommand attempts to figure out if the connection is going to use HTTPS and what port to use based on:

The protocol, if any, that you specify in URL.
The port, if any, that you specify in URL.
Whether you have provided certificate information for the request.

HttpCommand will use HTTPS if

You provide certificate information by using the Cert or PublicCertFile and PrivateKeyFile settings.
You specify 'https' as the protocol.
You don't specify a protocol, but do specify a port in URL of 443.

HttpCommand will use port 443 if you do not specify a port in URL and HttpCommand has otherwise determined from the above criteria that the connection should use HTTPS.

Encrypted Communications

In most cases, the use of certificates is hidden when using the HTTPS protocol because HttpCommand will automatically generate an "anonymous" certificate. This is sufficient to provide secure, encrypted, communications.

Authentication

Certificates can provide information about the identity of what is at the other end of the communications link.

Server Identity

When HttpCommand uses HTTPS, the PeerCert element of the result namespace contains the server's certificate. This can be used to validate the server is what you think it is. Most certificates will have a "chain" of parent certificates that show who issued the peer's certificate and any intermediate certificates. If the final certificate in the chain is issued by a Trusted Root Certificate Authority the domain listed in the certificate has undergone validation by an entity that you can trust.

Dyalog.com provides us with an example of a certificate chain:

      ⊢ r ← HttpCommand.Get 'https://dyalog.com'
[rc: 0 | msg:  | HTTP Status: 200 "OK" | ⍴Data: 23127]
      r.PeerCert  ⍝ Dyalog.com's certificate
 #.HttpCommand.[X509Cert]

      r.PeerCert.Chain
  #.HttpCommand.[X509Cert]  #.HttpCommand.[X509Cert]  #.HttpCommand.[X509Cert]  

     ⍪⊃r.PeerCert.Chain.Formatted.(Subject,' from ',Issuer)
 CN=*.dyalog.com from C=US,O=Let's Encrypt,CN=R3                                                              
 C=US,O=Let's Encrypt,CN=R3 from C=US,O=Internet Security Research Group,CN=ISRG Root X1                      
 C=US,O=Internet Security Research Group,CN=ISRG Root X1 from O=Digital Signature Trust Co.,CN=DST Root CA X3

This shows that the certificate for '*.dyalog.com' was issued by Let's Encrypt whose certificate was issued by Internet Security Research Group whose certificate was issued by Digital Signature Trust Co and Digital Signature Trust Co is a Trusted Root Certificate Authority.

The good news is that in most cases you'll never need to look at the certificate chain, but it is available in the rare event that you need to verify the server you're connecting to.

Client Identity

If the service you connect to uses client certificate authentication, you will need to use a certificate when sending your HTTP request. Generally this certificate will be issued by the service provider. A certificate consists of two parts - a public certificate file and a private key file. HttpCommand allows you to specify the certificate in two ways:

provide the file names for the public certificate file and private key file in the PublicCertFile and PrivateKeyFile settings respectively.
specify the Cert setting as either:
- An instance of Conga's X509Cert class
- a 2-element vector of the names of the public certificate and private key files

Other HTTPS Settings

HttpCommand has two more settings SSLFlags and Priority. The default values for these settings will suffice in almost all cases. If you need to use something other than the default, refer to the Conga User Guide.

Use of Conga

If your application has other components that also use Conga, it's advisable that each component have its own Conga "root" created by Conga.Init. This requires the Conga namespace from the conga workspace. The recommended setup is:

Copy the Conga namespace into your workspace. For this example, we will assume you've copied Conga into the root (#) namespace.
Explicitly point HttpCommand at this Conga namespace by setting HttpCommand.CongaRef←#.Conga (or wherever you've copied Conga).
All other Conga-using components should be similarly pointed at this Conga namespace.

When HttpCommand is initialized it will create a root named 'HttpCommand' when Conga.Init is called. Your other components will create their own Conga roots when they also call Conga.Init. This will ensure that the Conga shared libraries are loaded only once.

Deployment

In addition to the presence of other Conga-using components in your application, you need to consider how your application is deployed.

Dyalog Installation Folder is Available

If the Dyalog installation folder is available, and there are no other components that use Conga, you can use HttpCommand without any additional configuration. HttpCommand will copy the Conga namespace from the conga workspace in the Dyalog installation and find the shared libraries in the Dyalog APL installation folder.

Dyalog Installation Folder is Not Available

If the Dyalog installation folder is not available, for instance in the case of a runtime application, or a bound or stand-alone executable, then you will need to have a copy of the Conga namespace in your workspace and either put the Conga shared libraries in the same folder as your executuable or set CongaPath to the location of the shared libraries.

Ended: Advanced Topics

About ↵

MIT License

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Version 5.0

The major version bump from 4 to 5 was due to:
- swapping the meaning for the Timeout and WaitTime settings. Previously Timeout was used to indicate how long Conga's Wait function would wait for a response and WaitTime was how long HttpCommand would wait for a complete response before timing out.
- return codes and messages were normalized
Added new Auth and AuthType settings to more easily support token-based authentication.
Removed half-implemented support for streaming
Added GetHeader function to result namespace
More complete setting checking
Better handling of relative redirections
New documentation

HttpCommand

Upgrading to HttpCommand Version 5

Typical Usage Pattern

Philosophy

Dyalog Classes

Use of Conga

Further Reading

Usage ↵

Shortcut Methods

Get

GetJSON

Do

New

Arguments

Shortcut or Instance?

HttpCommand's Result

Typical Use in Code

Interacting with Web Services

Authentication/Authorization

HTTP Basic Authentication

Token-based Authentication

Interacting with JSON-based Web Services

Posting Data to a Web Service

Downloading a File

Twitter Example

First check rc and msg

Check HttpMessage and Data

Examine the request that HttpCommand will send

Debug Mode

Ended: Usage

Reference ↵

Settings ↵

Settings

Command

URL

Params

Headers

ContentType

Auth

AuthType

Cookies

Name/Value Pairs

Instance Settings

BufferSize

WaitTime

Cert

PublicCertFile

PrivateKeyFile

SSLFlags

Priority

Shared Settings

CongaPath

CongaRef

LDRC

CongaVersion

Instance settings

KeepAlive

MaxPayloadSize

MaxRedirections

Outfile

RequestOnly

SuppressHeaders

Timeout

TranslateData

Shared Settings

Debug

Ended: Settings

Shared Methods ↵

Get - issue an HTTP GET request

GetJSON - issue a request to a JSON-based web service

Do - issue a generic HTTP request

New - create a new instance of HttpCommand

Base64 Encoding

Base64Decode

Base64Encode

URL Encoding

UrlDecode

UrlEncode

Version

Documentation

Upgrading to `HttpCommand` Version 5

`Get`

`GetJSON`

`Do`

`New`

`HttpCommand`'s Result

First check `rc` and `msg`

Check `HttpMessage` and `Data`

Examine the request that `HttpCommand` will send

`Command`

`URL`

`Params`

`Headers`

`ContentType`

`Auth`

`AuthType`

`Cookies`

`BufferSize`

`WaitTime`

`Cert`

`PublicCertFile`

`PrivateKeyFile`

`SSLFlags`

`Priority`

`CongaPath`

`CongaRef`

`LDRC`

`CongaVersion`

`KeepAlive`

`MaxPayloadSize`

`MaxRedirections`

`Outfile`

`RequestOnly`

`SuppressHeaders`

`Timeout`

`TranslateData`

`Debug`

`Get` - issue an HTTP GET request

`GetJSON` - issue a request to a JSON-based web service

`Do` - issue a generic HTTP request

`New` - create a new instance of `HttpCommand`

`Base64Decode`

`Base64Encode`

`UrlDecode`

`UrlEncode`

`Version`

`Documentation`

`Upgrade`

`Run`

`Show`

`Config`

`Init`

`AddHeader`

`SetHeader`

`RemoveHeader`

`rc`

`msg`

`Elapsed`

`OutFile`

`BytesWritten`

`Command`

`Host`

`Path`

`Port`

`Secure`

`URL`

`HttpStatus`

`HttpMessage`

`HttpVersion`

`Data`

`Headers`

`∇GetHeader`

`Cookies`

`PeerCert`

`Redirections`

`rc` Values

`HttpCommand` Behavior