API question: protected methods in Python

Hello,

I’m wondering if anyone here has experience with using the Polaris API in Python and can help me debug.

I’m trying to use a protected method (RecordSetRecordsGet). When I run the script below with the url ‘https://catalog.onlib.org/PAPIService/REST/protected/v1/1033/100/1/none/recordsets/257846/records?userid=2376&wsid=943&startIndex=0&numRecords=100’ and api_access_key in line 41 (where I understood the AccessSecret should go) it succeeds about half the time (I reached out to Polaris support and they escalated this as a bug). But when I run it as written, passing the token and secret returned by the authenticator call, it always fails.

We are on version 7.8.

I don’t have a way to test python directly, but here is what chatgpt 5.6 sol says (below). If you continue to have issues, if you want to point to code repository or use the code blocking (triple ticks at start and finish ```) here on the forum, it might make further debugging a little easier.

The failure is caused by two signing errors in the Python code. The “works about half the time” behavior has a particularly strong explanation: urlsafe_b64encode() is incorrect for PAPI.

Polaris expects:

HMAC key: API access key
Signed text: HTTP method + exact URL + date + AccessSecret
Encoding: standard Base64

The screenshot instead uses the AccessSecret as the HMAC key and omits it from the signed text.

Use this implementation:

import base64
import hashlib
import hmac
from datetime import datetime, timezone
from email.utils import format_datetime


def papi_date():
    return format_datetime(datetime.now(timezone.utc), usegmt=True)


def make_signature(
    api_access_key,
    method,
    url,
    date,
    access_secret="",
):
    # Protected requests append AccessSecret to the signed text.
    message = f"{method}{url}{date}{access_secret}".encode("utf-8")

    digest = hmac.new(
        api_access_key.encode("utf-8"),  # Always the permanent API access key
        message,
        hashlib.sha1,
    ).digest()

    # PAPI requires standard Base64, not URL-safe Base64.
    return base64.b64encode(digest).decode("ascii")

The authentication request does not include an AccessSecret:

date = papi_date()

auth_url = (
    "https://catalog.onlib.org/PAPIService/REST/"
    "protected/v1/1033/100/1/authenticator/staff"
)

signature = make_signature(
    api_access_key,
    "POST",
    auth_url,
    date,
)

headers = {
    "Authorization": f"PWS {api_access_id}:{signature}",
    "PolarisDate": date,
    "Content-Type": "application/xml",
    "Accept": "application/json",
}

response = http.request(
    "POST",
    auth_url,
    body=body,
    headers=headers,
)

After retrieving the token and secret, sign the protected request like this:

auth_result = json.loads(response.data.decode("utf-8"))

access_token = auth_result["AccessToken"]
access_secret = auth_result["AccessSecret"]

url = (
    "https://catalog.onlib.org/PAPIService/REST/"
    f"protected/v1/1033/100/1/{access_token}/"
    "recordsets/257846/records"
    "?userid=2376&wsid=943&startIndex=0&numRecords=100"
)

date = papi_date()

signature = make_signature(
    api_access_key,
    "GET",
    url,
    date,
    access_secret,  # Appended to the message, not used as the HMAC key
)

headers = {
    "Authorization": f"PWS {api_access_id}:{signature}",
    "PolarisDate": date,
    "Accept": "application/json",
}

response = http.request("GET", url, headers=headers)

print(response.status)
print(response.data.decode("utf-8"))

Why the original sometimes succeeds:

base64.urlsafe_b64encode(...)

replaces + with - and / with _. Polaris expects ordinary Base64. For a SHA-1 digest, the URL-safe and standard encodings happen to be identical roughly 44% of the time—almost exactly the reported “half the time.” The official examples use standard Base64 and show signatures containing characters such as +. Polaris 7.8 authorization overview

Also:

  • Put the real AccessToken in the URI; do not use none.
  • Generate one date value per request and use that exact value in both the signature and PolarisDate header. Polaris requires RFC 1123 GMT and accepts only a ±30-minute window.
  • Sign the exact URL that is transmitted, including query order and encoding.
  • Pass headers directly to each request instead of mutating http.headers, which can retain stale request state.
  • Confirm that userid=2376 is the authenticated staff user’s PolarisUserID. A mismatched user can produce -8000 or -8001.
  • Polaris 7.8 can be configured for HMACSHA256; if the server’s KeyedHashAlgorithm is set to that, change hashlib.sha1 to hashlib.sha256.

The protected-method documentation explicitly says the token belongs in the URI and the AccessSecret is appended to the data being hashed. Protected-method signing instructions The endpoint itself requires userid, wsid, startIndex, and numRecords. RecordSetRecordsGet reference

Any success using /none/ is likely the separate Polaris server-side bug support has escalated; it should not be relied upon.

BTW, make sure to use the copy code helper on any of these code blocks:

There are some that span more than is visible so it will make sure you get the whole code sample.

I do think updating the base 64 encoding from urlsafe_b64encode has helped with the public method in my test, but I’m still getting no results for the protected method. I’m uploading what I have - I completely understand that people are busy and may not have time to take a look, but I would prefer that this not be fed to generative AI.

from email.utils import formatdate
import hmac
import hashlib
import base64
import certifi
import urllib3
import json


def make_digest(message, key):
    key = bytes(key, 'UTF-8')
    message = bytes(message, 'UTF-8')
    digester = hmac.new(key, message, hashlib.sha1)
    signature1 = digester.digest()
    signature2 = (str(base64.b64encode(signature1))[2:-1]).strip("")
    return signature2

def auth_staff(base_url,http,api_access_key,api_access_id,body,date):
    url=base_url+'/PAPIService/REST/protected/v1/1033/100/1/authenticator/staff'
    message = "POST" + url + str(date)
    myHash = make_digest(message, api_access_key)
    headers = {'Authorization':"PWS {staffApiID}:{signature}".format(staffApiID=api_access_id,signature=myHash),
                'PolarisDate':str(date),
                'Content-Type':'application/xml',
                'Accept':'application/json'}
    response = http.request('POST',url, body=body,headers=headers)
    return json.loads(response.data.decode('utf-8'))["AccessToken"],json.loads(response.data.decode('utf-8'))["AccessSecret"]

def protected_method(base_url,http,api_access_key,api_access_id,body,date):
    retries_limit=3
    retries_count=0

    token,key=auth_staff(base_url,http,api_access_key,api_access_id,body,date)
    # url should use the correct record set ID/username/workstation
    url=base_url+'/PAPIService/REST/protected/v1/1033/100/1/'+token+'/recordsets/257846/records?userid=2376&wsid=943&startIndex=0&numRecords=100'
    message = "GET" + url + str(date)
    myHash = make_digest(message, key)
    headers = {'Authorization':"PWS {staffApiID}:{signature}".format(staffApiID=api_access_id,signature=myHash),
                    'PolarisDate':date,
                    'Accept':'application/json'}
    response = http.request('GET',url,headers=headers)
    print(json.loads(response.data.decode('utf-8')))


def public_method(base_url,http,api_access_key,api_access_id,body,date):
    date = formatdate(timeval=None, localtime=False, usegmt=True)
    # update the bib record ID here
    url=base_url+'/PAPIService/REST/public/v1/1033/100/1/bib/2291044/holdings'
    message = "GET" + url + str(date)
    myHash = make_digest(message, api_access_key)
    headers = {'Authorization':"PWS {staffApiID}:{signature}".format(staffApiID=api_access_id,signature=myHash),
                'PolarisDate':str(date),
                'Content-Type':'application/xml',
                'Accept':'application/json'}
    response = http.request('GET',url,headers=headers)
    print(json.loads(response.data.decode('utf-8')))


def main():
    http=urllib3.PoolManager(
        cert_reqs="CERT_REQUIRED",
        ca_certs=certifi.where()
    )
    ## your info here
    base_url=''
    api_access_key=''
    api_access_id=''
    polaris_username=''
    polaris_password=''
    domain=''
    body = f"<?xml version='1.0' encoding='UTF-8'?><AuthenticationData><Domain>{domain}</Domain><Username>{polaris_username}</Username><Password>{polaris_password}</Password></AuthenticationData>"
    date = formatdate(timeval=None, localtime=False, usegmt=True)

    print("Polaris API methods test",end="\n\n")
    print(f"API access key: {api_access_key} \nAPI access ID: {api_access_id} \nPolaris username/pw: {polaris_username}/{polaris_password} \nBody: {body} \nDate: {date}\n")

    print('Public method BibHoldingsGet: looks for holdings on bib 2291044')
    public_method(base_url,http,api_access_key,api_access_id,body,date)

    print('Protected method RecordSetRecordsGet: lists record IDs in record set 257846')
    protected_method(base_url,http,api_access_key,api_access_id,body,date)
    
main()

Hi Eleanor,

First thing that stands out to me is the headers. When you are using a protected method, you need the “X PAPI AccessToken” header. I have attached a snippet from the documentation below. Try including that and let me know.

For anyone wondering, this is what I heard from Polaris support/development: “as long as there is the authentication in the header the token in the URI is not required. I believe the place needs to exist, and that is why having “none” there does allow for the API call to go through.” So the AuthenticateStaffUser method isn’t actually required before calling protected methods. My understanding is that the documentation will be updated to reflect this.

Hello Aaron,

Thanks for taking a look. I think the snippet you attached just applies to authenticating patrons, not to using protected methods in general. When I added ‘X PAPI AccessToken’ to the headers in the RecordSetRecordsGet call, it failed.

Regards,
Eleanor

You might also see if this python based PAPI programming library has any clues: PolarisAPI/polaris.py at master · DarienLibrary/PolarisAPI · GitHub

Hi Eleanor,

You do need that header whether you are calling as a patron or staff member. You also need to include the dashes - ‘X-PAPI-AccessToken’

1 Like