Overview

The Enterprise Fact Extraction service of the Text Analysis Fact Extraction package is part of a series of enterprise-grade, natural-language products that find domain-specific relationships between entities in input texts. The package's Enterprise Fact Extraction service identies enterprise events to track what’s happening in the business world. Are there any changes in senior management at a company of interest? Who is releasing new products? Which acquisitions are in the works?

The service can monitor these activities by identifying membership information such as a person’s affiliations, management changes, product releases, mergers and acquisitions, and organizational information such as founder, location, or contact information.

Use Case

An example use of the service is an application that scans corporate press releases in multiple formats, such as PDF, Microsoft Word, RTF, and HTML. Your application can extract, from those documents, information about company acquisitions and plot a graph of which companies acquired others and for how much.

You can pass the press release documents to the Enterprise Fact Extraction service, which automatically determines the file format, extracts the text, determines the language in which they are written, and returns any entities of a corporate nature contained in the press releases.

You can identify events to save, for example saving only the acquisition events where a price is mentioned, indicated by entities of the type BuyEvent that have a child entity of type Price. In addition to the price, you can save the names of the companies involved. For example, if a press release contains the sentences "Muttski Enterprises acquired Bow-Wow Technologies for $13bn. Buddy Coddo was promoted to Vice President of Operations.", you can save only the following child entities of the BuyEvent type found in the first sentence:

Text

Muttski Enterprises

Bow-Wow Technologies

$13bn

Entity Type

OrganizationA

OrganizationB

Price

Description

The buyer

The acquired company

The price OrganizationA paid for OrganizationB




The service supports text in English.

The service accepts input in a wide variety of formats:

  • Abobe PDF
  • Generic email messages (.eml)
  • HTML
  • Microsoft Excel
  • Microsoft Outlook email messages (.msg)
  • Microsoft PowerPoint
  • Microsoft Word
  • Open Document Presentation
  • Open Document Spreadsheet
  • Open Document Text
  • Plain Text
  • Rich Text Format (RTF)
  • WordPerfect
  • XML

The size of each input file is limited to 100 kB.

The tenant parameter is not required because the service is stateless and no data is persisted.


API Reference

/

/

post

Extract entities related to enterprises (businesses), such as personal affiliations, product releases, and mergers & acquisitions, in English input documents. Detailed explanations of the enterprise entities that text analysis can extract are available in the Enterprise Fact Extraction section of the SAP HANA Text Analysis Language Reference Guide.


An empty entities array is normal

It is not an error if the service returns an empty entities array. Not all text contains entities as defined and recognized by the service. For example, the English sentence "It's the end of the world as we know it, and I feel fine" contains no entities, nor do any of these translations:

Spanish:

German:

Korean:

Russian:

Es el fin del mundo tal como lo conocemos, y me siento bien.

Es ist das Ende der Welt, wie wir es kennen, und ich fühle mich gut.

우리가 알고있는대로 그것은 세상의 종말이며, 나는 기분이 좋아집니다.

Это конец света, как мы его знаем, и я чувствую себя прекрасно.


The label and labelPath members

Some of the entity types that the service identifies are given general categories and then subdivided into more specific classifications. For example, URI is a general category with the subtypes EMAIL, IP, and URL. The label member of the entities array is the most specific entity type of an extracted entity. The labelPath member includes the general category and the subtype, separated by a forward slash ("/").
That means, if a document contains the web address "http://www.sap.com" in its text, that string is extracted as an entity with its label attribute's value set to URL and its labelPath set to URI/URL.
If the entity type does not have subtypes, for example, PERSON, the label and labelPath values are identical.


How parent and child entities are linked in the JSON response

Some entities are composed of other entities. This hierarchical relationship is indicated in the JSON response by an extra attribute, parent, that appears only on child entities. It contains an integer value that matches the id value of the parent.

The parent entity always appears before its children in the entities array. The order in which children appear is not guaranteed.


Default Language

The default language is either the first value listed in the languageCodes input parameter (see Setting a subset of languages in this topic) or English if the languageCodes input parameter is not specified.


Setting a subset of languages

You can instruct the service to choose from a specific, reduced set of languages by setting the languageCodes input parameter. This forces the service to choose from one of the languages you supply.
Use this setting with caution. If, for example, you set languageCodes to Danish, German, or Dutch and the input text is in Russian, the service cannot return Russian. It must return the default.


Meaning of the textSize value

The returned attribute textSize represents the amount of character data in the input, not the number of bytes. If the input is in plain text file without accented characters, textSize equals the input file's size. However, if the input is a binary file such as a PDF or Microsoft Word document, the textSize will probably be much smaller than the file size, especially if the file contains a lot of non-textual data such as an embedded image.


Annotated JSON schema

The JSON schema contains the descriptions of the objects and members of the JSON response that the Entity Extraction service returns. To read the schema, click the POST link in the API Reference, then click the RESPONSE tab.


Further references

You can find extensive details on the capabilities and behavior of SAP's entity extraction technology in the Entity Extraction chapter of the SAP HANA Text Analysis Language Reference Guide (PDF).


Python Tutorial

This tutorial mirrors the use case described in the Overview. This tutorial shows how to retrieve and store entities. It does not illustrate the plotting phase of the application described in the use case.

Plot corporate acquisitions

In this tutorial, you are using the Enterprise Fact Extraction service to extract corporate acquisitions from press releases in a variety of formats such as PDF, Microsoft Word, RTF, and HTML. Your app plots a graph showing who acquired whom and for how much.

Get an access token

To use the service, you must pass an access token in each call. Get the token from the OAuth2 service.

import requests
import json

# Replace the two following values with your client secret and client ID.
client_secret = 'clientSecretPlaceholder'
client_id = 'clientIDPlaceholder'

s = requests.Session()

# Get the access token from the OAuth2 service.
auth_url = 'https://api.beta.yaas.io/hybris/oauth2/v1/token'
r = s.post(auth_url, data= {'client_secret':client_secret, 'client_id':client_id,'grant_type':'client_credentials'})
access_token = r.json()['access_token']

Call the service

The POST request body for this service includes a single value: the text on which to perform enterprise fact extraction. The variable press_release contains the raw binary data read from a press release file.

# The Enterprise Fact Extraction service's URL
service_url = 'https://api.beta.yaas.io/sap/ta-enterprisefacts/v1/'

# HTTP request headers
req_headers = {}

# Set content-type to 'application/octet-stream'. The service automatically
# determines the content type is HTML and removes the markup prior to performing sentiment 
# analysis.
req_headers['content-type'] = 'application/octet-stream'
req_headers['Cache-Control'] = 'no-cache'
req_headers['Connection'] = 'keep-alive'
req_headers['Accept-Encoding'] = 'gzip'
req_headers['Authorization'] = 'Bearer {}'.format(access_token)

# Make the REST call to the Enterprise Fact Extraction service. Pass the binary 
# data in raw form. Do not base64-encode the data.
response = s.post(url = service_url,  headers = req_headers, data = press_release)


Taking the sample text from the use case presented in the Overview, , the JSON response from the service starts with the following:

{
    "mimeType": "text/plain", 
    "entities": [
        {
            "sentence": 1, 
            "text": "Muttski Enterprises acquired Bow-Wow Technologies for $13bn", 
            "label": "BuyEvent", 
            "paragraph": 1, 
            "offset": 0, 
            "normalizedForm": "", 
            "id": 1, 
            "labelPath": "BuyEvent"
        }, 
        {
            "parent": 1, 
            "sentence": 1, 
            "text": "Muttski Enterprises", 
            "label": "OrganizationA", 
            "paragraph": 1, 
            "offset": 0, 
            "normalizedForm": "", 
            "id": 2, 
            "labelPath": "OrganizationA"
        }, 

The last few lines of the response are:

        {
            "parent": 13, 
            "sentence": 2, 
            "text": "Operations", 
            "label": "Organization", 
            "paragraph": 1, 
            "offset": 107, 
            "normalizedForm": "", 
            "id": 18, 
            "labelPath": "Organization"
        }
    ], 
    "textSize": 119, 
    "language": "en"
}

Each entity extracted from the input text appears in the response's entities array in order of appearance. Enterprise entities have a hierarchical (parent←→child) relationship. This diagram illustrates how the entities in this tutorial's example text are related to one another:

graph TD; root1(BuyEvent

Muttski Enterprises acquired Bow-Wow Technologies for $13bn)-->orgA(OrganizationA

Muttski Enterprises); root1-->action1(Action

acquired); root1-->orgB(OrganizationA

Bow-Wow Technologies); root1-->price(Price

$13bn); root2(HireEvent

Buddy Coddo was promoted to Vice President of Operations)-->person(Person

Buddy Coddo); root2-->action2(Action

promoted); root2-->position(Position

Vice President); root2-->organization(Organization

Operations);

classDef blackBox fill::#000000; class root1,root2,orgA,orgB,price,action1,action2,person,position,organization blackBox;

Every entity returned has at least eight attributes: sentence, text, label, paragraph, offset, normalizedForm, id, and labelPath. For a detailed description of each attribute in the response, see the link to the JSON schema in the Details section of this service.

Some entities have a ninth attribute: parent, which associates child entities with the parents they comprise.

You care only about acquisition events in which a price is mentioned, such as those whose label attribute is BuyEvent with a child whose label is Price. See the Details section for a link to a list of all Enterprise entity types. You store only the OrganizationA, OrganizationB, and Price child entities of BuyEvents.

Thus, given the example text, you store three of the BuyEvent entity's children, but you do not store the HireEvent entity nor anything under it.

The method store_acquisition is defined in your application; its code is not included here.

# Process the result
if response.status_code == 200:
    # De-serialize the JSON reply and get the entities list.
    response_dict = json.loads(response.text)
    # If the service returns no entities, it is not an error, it just means no
    # enterprise entities are contained in the text. 
    # Thus, the 2nd parameter of the get() call is left out.
    entities = response_dict.get('entities')

    found_one_buyevent = None  # found at least 1 action in the response

    for e in entities:
        entity_type = e.get('label')
        # At a BuyEvent? They always precede their children in the
        # entity array.
        if (entity_type == 'BuyEvent'):
            # If already encountered another BuyEvent in the array, 
            # now's the time to save it.
            if (found_one_buyevent and curr_price != ''):
                store_acquisition(curr_org_a, curr_org_b, curr_price)

            found_one_buyevent = True
            curr_buyevent_id = e.get('id')
            curr_price = ''
        elif (found_one_buyevent and e.get('parent') == curr_buyevent_id):
            # This is a child entity of the current BuyEvent. Save it if it's
            # either OrganizationA, OrganizationB, or Price.
            if (e.get('label') == 'OrganizationA'):
                curr_org_a = e.get('text')
            elif (e.get('label') == 'OrganizationB'):
                curr_org_b = e.get('text')
            elif (e.get('label') == 'Price'):
                curr_price = e.get('text')

    # End of entities array. If found any, the last one still needs to be
    # stored.
    if (found_one_buyevent and curr_price != ''):
        store_acquisition(curr_org_a, curr_org_b, curr_price)
else:
    print 'Error', response.status_code
    print response.text


  • Send feedback

    If you find any information that is unclear or incorrect, please let us know so that we can improve the Dev Portal content.

  • Get Help

    Use our private help channel. Receive updates over email and contact our specialists directly.

  • hybris Experts

    If you need more information about this topic, visit hybris Experts to post your own question and interact with our community and experts.