Centralized Probe Management
  • 11 Dec 2023
  • 13 Minutes to read
  • Dark
    Light
  • PDF

Centralized Probe Management

  • Dark
    Light
  • PDF

Article Summary

About this Article
This article is intended to highlight the features of the Centralized Probe Management (CPM) API. API is compatible with Version 2.9 and Above - CMS and Probes.
The Install Probe, Upgrade Probe, Uninstall Probe, Install Script and vsp-cli command APIs are available only for Version 2.11 and Above.


Centralized Probe Management (CPM) API aims to simplify the upgrade and troubleshooting requirements scenarios on Probes, deployed on different application platforms. Without this API, it becomes necessary to log in to the host for upgrade/ troubleshooting.


Sample API Usage:  

curl 'https://<CMS_IP_Address>/rms/probes' -H 'Authorization: Basic <base64_encode_of_userid:password>' --insecure


Header:

Content-type: application/json

Authorization: Basic <base64 encode of userid:password>


Installation

Method 1: Script-based Installation

CPM installation can be performed using the Probe Installable – vsp_install_vm.sh (Linux) OR vsp_install_vm.bat (Windows). Utilize the below parameters as required:

ParameterDescription
-mTo install CPM. Allowed values are 1 (Enable), 0 (Default - Disable)
-WTo uninstall CPM. Allowed values are 1 (Enable), 0 (Default - Disable)
-wTo uninstall the existing CPM version and install a new version available on LFR. Allowed values are 1 (Enable), 0 (Default - Disable)
NOTE
Whenever these parameters are utilized, ensure that LFR IP address is provided with the -l parameter so that the latest version of the script from LFR is utilized.

Click Linux Unsigned / Linux Signed or Windows  2008 and Above / Windows 2003 for more information about Script-based Probe Installation.


Method 2: MSI-based Quiet Installation (Windows)

CPM installation is performed using the quiet installation method using MSI for Windows. Utilize the below parameters as required:

ParameterDescription
START_CPM_SERVICETo start/stop CPM services after Installation. Allowed Values are: 1 (Start; Default) and (Stop)
NOTE
Whenever these parameters are utilized, ensure that LFR IP address is provided with the parameter "LFR_IP" so that the latest version of the script from LFR is utilized.


Command for CPM Installation:

start /wait msiexec /i VSPCPM.msi /qn /norestart CMS_IP=<CMS_IP> ACCEPTEULA=1 /l*v c:\vspcpm_installermsi.log


Command for CPM Uninstallation:

start /wait msiexec /x {38276D79-4FD0-40B8-A120-0C49D5591A3C} /quiet /qn /norestart /l*v c:\vspcpm_installermsi.log


Click MSI Installation for more information about MSI-based Probe Installation for Windows.


Verification

Utilize the below commands to view the status of the CPM service.

  1. Linux: 
    sudo service vspcpm status
  2. Windows: 
    sc query vspcpm


Data Directories

  1. The extracted API information is stored in the mounted CPMS data directory, that can be accessed at /home/virsec/cpmdata
  2. Files are automatically purged after a configured duration. By default, this duration is 7 days


Procure List of Probes

The below API URL provides information related to the Probes. Only a Super Admin user can utilize this API.

Request Type: GET 

URL: https://<CMS_IP_Address>/rms/probes 

Response Code:

200 - Success

Sample Response:

[

    {

        "identifier": "d0528643-21ae-4fe3-9067-58693893e13f",

        "ipaddr": "10.15.31.15",

        "hostname": "ubuntu16.virsec.local",

        "vsp_version": "2.6.0",

        "status": "ACTIVE"

    },

    {

        "identifier": "3c0fae9a-0075-4e8d-8d90-59ad94d0fb12",

        "ipaddr": "10.15.31.17",

        "hostname": "ubuntu16.virsec.local",

        "vsp_version": "dakota.qa2",

        "status": "ACTIVE"

    }

]


Procure Logs

The below API procures logs from the Probe instances:

URL/JSONRequest TypeDescription
URL:
https://<CMS_IP_Address>/rms/fetch/log/all
[Deprecated from VSP 2.11 onwards]
GETFetches logs from all the Probes
URL:
https://<CMS_IP_Address>/rms/fetch/log/{probeid}
GETFetches log from a specific probe. Probe IDs can be procured from https://<CMS_IP_Address>/rms/probes as described in Procure List of Probes

URL:
https://<CMS_IP_Address>/rms/fetch/log

JSON:

{
    "probes" : [
               "PROBE_ID1", 
               "PROBE_ID2"
    ]
}

POSTFetches log from a group of probes. Probe IDs can be procured from https://<CMS_IP_Address>/rms/probes as described in Procure List of Probes

Response Code:

200 - Success

Sample Responses:

Response containing link to the status URL:

{

    "status": "SUCCESS",

    "message": "log-file fetch started, view status at url - /rms/status/cmd_489d40a5-85dd-4316-a06e-073e23223aca",

    "url": "/rms/status/cmd_489d40a5-85dd-4316-a06e-073e23223aca",

    "api": "Fetch log"

}


Response containing status of all the probes along with the download path:

{

    "status": "SUCCESS",

    "api": "Fetch log",

    "probeList": [

        {

            "host": "ubuntu16",

            "ip": "10.15.50.15",

            "identifier": "1f96fdb6-cd50-4558-a315-530e8038b1b5",

            "downloadUrl": "/rms/download/1f96fdb6-cd50-4558-a315-530e8038b1b5/vsp_log_ubuntu16_2023-07-26.tar.gz",

            "status": "SUCCESS"

        }

    ]

}

In a successful response, the download URL link is provided for each Probe. The log files can be downloaded once the status changes to “SUCCESS”. If not, access/refresh the URL after some time to ensure that the log collection is complete and the status changes to “SUCCESS”.


Procure Statistics

The below API procures statistics from the Probe instances:

URL/JSONRequest TypeDescription
URL:
https://<CMS_IP_Address>/rms/fetch/stats/all
[Deprecated from VSP 2.11 onwards]
GETFetches statistics from all the Probes
URL:
https://<CMS_IP_Address>/rms/fetch/stats/{probeid}
GETFetches statistics from a specific probe. Probe IDs can be procured from https://<CMS_IP_Address>/rms/probes as described in Procure List of Probes

URL:
https://<CMS_IP_Address>/rms/fetch/stats

JSON:

{
    "probes" : [
               "PROBE_ID1", 
               "PROBE_ID2"
    ]
}

POSTFetches statistics from a group of probes. Probe IDs can be procured from https://<CMS_IP_Address>/rms/probes as described in Procure List of Probes

Response Code:

200 - Success

Sample Response:

Response containing a link to the status URL:

{

    "status": "SUCCESS",

    "message": "stats-file fetch started, view status at url - /rms/status/cmd_489d40a5-85dd-4316-a06e-073e23223aca",

    "url": "/rms/status/cmd_489d40a5-85dd-4316-a06e-073e23223aca",

    "api": "Fetch stats"

 }

In a successful response, the download URL link is provided for each Probe. The statistics files can be downloaded once the status changes to “SUCCESS”. If not, access/refresh the URL after some time to ensure that the statistics collection is complete and the status changes to “SUCCESS”.


Procure Data

The below API procures data from the Probe instances:

URL/JSONRequest TypeDescription
URL:
https://<CMS_IP_Address>/rms/fetch/data/all
[Deprecated from VSP 2.11 onwards]
GETFetches data from all the Probes
URL:
https://<CMS_IP_Address>/rms/fetch/data/{probeid}
GETFetches data from a specific probe. Probe IDs can be procured from https://<CMS_IP_Address>/rms/probes as described in Procure List of Probes

URL:
https://<CMS_IP_Address>/rms/fetch/data

JSON:

{
    "probes" : [
               "PROBE_ID1", 
               "PROBE_ID2"
    ]
}

POSTFetches data from a group of probes. Probe IDs can be procured from https://<CMS_IP_Address>/rms/probes as described in Procure List of Probes

Response Code:

200 - Success

Sample Response:

Response containing a link to the status URL:

{

    "status": "SUCCESS",

    "message": "data-file fetch started, view data at url - /rms/status/cmd_489d40a5-85dd-4316-a06e-073e23223aca",

    "url": "/rms/data/cmd_489d40a5-85dd-4316-a06e-073e23223aca",

    "api": "Fetch data"

 }

In a successful response, the download URL link is provided for each Probe. The data files can be downloaded once the status changes to “SUCCESS”. If not, access/refresh the URL after some time to ensure that the data collection is complete and the status changes to “SUCCESS”.


Modify Log Levels

Modify the log levels using the below information. Log level modifications are supported on individual probes only. Allowed log levels are - debug, info, trace.

URL/JSONDescription
URL:
https://<CMS_IP_Address>/rms/enable/log/{probeid}/{moduleid}/{log-level}?type=(<Type>)&password=base64 encoded
  • (Optional) Type – Indicates the type of log file:
    • State log - Module state logs
    • Message log - Module's interface message specific log
    • Incident log - Log related to incident generated by module
    • All modules have State log, but may not have message and incident logs. Currently, message and incident logs are available for AE and HMM modules
  • Base64 encoded password to be provided in case vsp-cli is password protected on the probe machine
  • Query string is an optional field. If this field is not used, then, state logs are enabled

URL:
https://<CMS_IP_Address>/rms/enable/log/{probeid}/all/{log-level}
  • Modifies the log level across all modules
  • Example:  /enable/log/{probeid}/all/info –changes the log level of all modules to the defined log-level for the Probe

URL:
https://<CMS_IP_Address>/rms/ /enable/log/{probeid}/{moduleid}/{log-level}
  • Modified log level on a specific probe for a given module
  • Allowed Module ID:
    • ae – vRule Engine
    • fde – Framework Discovery Engine
    • fsm – File System Monitoring
    • rmp – Memory Exploit Protection
    • vipcs – vIPC Server
    • vma - vsp-memory-assist
    • aep – vRule Proxy
    • weba – web-assist
    • vmgr – VSP Manager
    • all – All the probe modules
  • Actual vsp-cli module IDs are also allowed

Sample Response:

Modify the log level for a specific module:

{

    "status": "SUCCESS",

    "logMessage": "Setting log level, view status at url - /rms/status/cmd_230ead5f-b2ce-40c0-9533-370a5476b564",

    "url": "/rms/status/cmd_230ead5f-b2ce-40c0-9533-370a5476b564",

    "apiCommand": "change loglevel to debug , for module ae"

}


Download Files

The API lists all the files available on the probe.

URLMethodDescription
https://<CMS_IP_Address>/rms/download/{probeid}GETList all available fetched files such logs/stats or data from a specific probe identified by probeid
https://<CMS_IP_Address>/rms/downloadGETList all available fetched files such logs/stats or data from all probes

Response Code:

200 - Success

Sample Responses: 

Download file list from the required probe:

[

    {

        "identifier": "1f96fdb6-cd50-4558-a315-530e8038b1b5",

        "name": "vsp_data_ubuntu16_2023-07-26.zip",

        "probeid": "1f96fdb6-cd50-4558-a315-530e8038b1b5",

        "url": "/rms/download/1f96fdb6-cd50-4558-a315-530e8038b1b5/vsp_data_ubuntu16_2023-07-26.zip"

    }

]

Download file list from all probes:

[

    {

        "identifier": "1f96fdb6-cd50-4558-a315-530e8038b1b5",

        "name": "vsp_data_ubuntu16_2023-07-26.zip",

        "probeid": "1f96fdb6-cd50-4558-a315-530e8038b1b5",

        "url": "/rms/download/1f96fdb6-cd50-4558-a315-530e8038b1b5/vsp_data_ubuntu16_2023-07-26.zip"

    },

    {

        "identifier": "1f96fdb6-cd50-4558-a315-530e8038b1b5",

        "name": "vsp_stats_ubuntu16_2023-07-25.zip",

        "probeid": "1f96fdb6-cd50-4558-a315-530e8038b1b5",

        "url": "/rms/download/1f96fdb6-cd50-4558-a315-530e8038b1b5/vsp_stats_ubuntu16_2023-07-25.zip"

    },

    {

        "identifier": "d71d0fbf-f318-4519-bf12-facca50e9607",

        "name": "vsp_log_ks2050_2023-07-26.tar.gz",

        "probeid": "d71d0fbf-f318-4519-bf12-facca50e9607",

        "url": "/rms/download/d71d0fbf-f318-4519-bf12-facca50e9607/vsp_log_ks2050_2023-07-26.tar.gz"

    },

]


View Commands

This API lists all the requests submitted to this API with the status of completion.  

Accepted Parameters:

  1. from - Integer “n” can be provided as the optional parameter in “from” to view all the API requests submitted for the past “n” hours. If the parameter “from” is not provided or is invalid, all the API requests submitted are provided
  2. cmdpage – Integer “i” can be provided as the optional parameter in “cmdpage” to specify the number of commands to be displayed per page
  3. probepage - Integer “j” can be provided as the optional parameter in “probepage” to specify the number of probes to be displayed per page

 

Request Type: GET 

URL: https://<CMS_IP_Address>/rms/view/api?from={n}&cmdpage={i}&probepage={j} 

  

Sample Response:

{

        "apiRequest": "Fetch Data",

        "commandUuid": "cmd_a5929978-16c1-4489-a52d-e9fd8c65caea",

        "timestamp": "2023-07-06T11:34:11.91",

        "commandStatus": "SUCCESS",

        "probeStatusList": {

            "8aa9017b-b347-404a-8d4a-0dc90342a253": {

                "tenentId": "default",

                "ipaddr": "172.30.208.1",

                "hostname": "host.docker.internal",

                "version": "Not Installed",

                "identifier": "8aa9017b-b347-404a-8d4a-0dc90342a253",

                "downloadUrl": "/rms/download/8aa9017b-b347-404a-8d4a-0dc90342a253",

                "executionStatusTimestamp": "2023-07-06T11:34:11.91",

                "status": "SUCCESS",

                "logbuffer": "\nhost.docker.internal ->  Status: SUCCESS  Reason: \nhost.docker.internal ->\tCommand successfully complete",

                "errorbuffer": ""

            }

      }

}


Install Probe

This API is available only for Version 2.11 and Above.

Probe installation can be performed using the below information: 

Accepted Parameters: All the parameters are optional. If they are not provided, the default values are utilized by the Probe installable 

  1. cmsdns – DNS name for CMS
  2. kafkadns - DNS name for Kafka
  3. remoteae true (for Remote vRule Engine) OR false (for Embedded vRule Engine)
  4. hostentry – Entries are not added in the file /etc/hosts to resolve the LFR and CMS DNS names. Allowed values are: true OR false
  5. password – Base64 encoded password
  6. sku – (Linux only) Required SKU. Allowed values are: web, host OR mem. Default value is web (if not specified)
  7. hostname – Hostname of the Probe
  8. hostProfileTag - Host Profile Tag allows for an application instance to be auto-associated with a process profile on CMS
  9. appServiceTags - List of app service tags separated by a comma; App service tags enable an Application Instance to be auto-associated with an application on CMS
  10. hostonly - (Windows only) To install HOST-ONLY features. Allowed Values are 0 (complete installation) or 1 (Host only installation)


URL/JSONRequest TypeDescription
URL:
https://<CMS_IP_Address>/rms/install/{probeid}?cmsdns=<dns>&kafkadns=<dns>&remoteae=<true|false>&hostentry=<true|false>&password=<base64 encoded>&sku=<web|host|mem>&hostname=<hostname>&hostProfileTag=<host profile tag>&appServiceTags=ast1,ast2&hostonly=<true | false>
GETA specific Probe is installed to the Version present on LFR
URL:
https://<CMS_IP_Address>/rms/install 


Request JSON:

{
"cmsdns" : "<Custom CMS DNS Name>",
"kafkadns" : "<Custom Kafka DNS Name>",
"remoteae" : false,
"hostentry" : true,
"version" : "latest",
"password" : "<base 64 econded>",
"sku" : "<web | host | mem>"
"hostProfileTag" : "hpt1",
"appServiceTags" : ["ast1","ast2"],
"hostOnly" : true,
"probeSet":[
    "PROBE_ID1", 
    "PROBE_ID2"
  ]
}

POSTInstallation on specific Probes connected with CMS. The provided parameter values are common for all the mentioned Probes. If the values differ, ensure that separate API calls are made for the corresponding probes

Response Code:

200 - Success

 Sample Response: Single Probe Installation (GET)

{

"status": "SUCCESS",

"message": "install started, view status at url - /rms/status/cmd_0c64fb24-0bd8-4d1c-947d-f30483fafd02",

"url": "/rms/status/cmd_0c64fb24-0bd8-4d1c-947d-f30483fafd02",

"api": "install probe",

"version": "latest"

}


Upgrade Probe

This API is available only for Version 2.11 and Above.

Probe upgrade can be performed using the below information:

NOTE
sku modifications during upgrade is not supported.


Accepted Parameters: All the parameters are optional. If they are not provided, the default values are utilized by the Probe installable

  1. version – VSP version to which the Probe must be upgraded. This version must be available on LFR. Currently, only the value “latest” is allowed as LFR contains only one VSP version
  2. cmsdns – Custom DNS name for CMS. Ensure that the value provided during upgrade is the same as the value provided during installation
  3. kafkadns – Custom DNS name for Kafka. Ensure that the value provided during upgrade is the same as the value provided during installation
  4. remoteae true (for Remote vRule Engine) OR false (for Embedded vRule Engine)
  5. hostentry – Entries are not added in the file /etc/hosts to resolve the LFR and CMS DNS names. Allowed values are: true OR false
  6. password – Base64 encoded password
URL/JSONRequest TypeDescription
URL:
https://<CMS_IP_Address>/rms/upgrade/{probeid}?version="latest"&cmsdns=<dns>&kafkadns=<dns>&remoteae=<true|false>&hostonly=<true|false>&password=<base64 encoded>
GETA specific Probe is installed to the Version present on LFR
URL:
https://<CMS_IP_Address>/rms/upgrade/all?version="latest"&cmsdns=<dns>&kafkadns=<dns>&remoteae=<true|false>&hostonly=<true|false>&password=<base64 encoded>
GETAll Probes registered with CMS are upgraded to the provided Version present on LFR
URL:
https://<CMS_IP_Address>/rms/upgrade 


Request JSON:

{
"cmsdns" : "<Custom CMS DNS Name>",
"kafkadns" : "<Custom Kafka DNS Name>",
"remoteae" : false,
"hostentry" : true,
"version" : "latest",
"password" : "<base 64 econded>"
"probeSet":[
    "PROBE_ID1", 
    "PROBE_ID2"
  ]
}

POSTSpecific Probes are upgraded to the provided Version present on LFR. The provided parameter values are common for all the mentioned Probes. If the values differ, ensure that separate API calls are made for the corresponding probes

Response Code:

200 - Success

 Sample Response: Single Probe Upgrade (GET)

"status": "SUCCESS", 

"message": "upgrade started, view status at url - /rms/status/cmd_0c64fb24-0bd8-4d1c-947d-f30483fafd02", 

"url": "/rms/status/cmd_0c64fb24-0bd8-4d1c-947d-f30483fafd02", 

"api": "upgrade probe", 

"version": "2.11.0"

}


Uninstall Probe

This API is available only for Version 2.11 and Above.

Probe installation can be performed using the below information:

 Accepted Parameters:

  1. password – Base64 encoded password
URL/JSONRequest TypeDescription
URL:
https://<CMS_IP_Address>/rms/uninstall/{probeid}?password=<base64 encoded>
GETA specific Probe is uninstalled
URL:
https://<CMS_IP_Address>/rms/uninstall 


Request JSON:

{
"password" : "<base 64 econded>",
"probeSet":[
    "PROBE_ID1", 
    "PROBE_ID2"
  ]
}

POSTUn-installation on specific Probes connected with CMS

Response Code:

200 - Success

 Sample Response: Single Probe Uninstallation (GET)

{

"status": "SUCCESS",

"message": "uninstall started, view status at url - /rms/status/cmd_0c64fb24-0bd8-4d1c-947d-f30483fafd02",

"url": "/rms/status/cmd_0c64fb24-0bd8-4d1c-947d-f30483fafd02",

"api": "uninstall probe",

"version": "latest"

}


Install Script

This API is available only for Version 2.11 and Above. It enables installation of a script from LFR to a Probe machine. 


Accepted Parameters:

  1. sourcePath – (Required) Specify the relative source path of the script on LFR. It must specify a file. Example: If the LFR path is: https://<LFR_IPAddress>:8443/vsp/pas1/pa1.sh then source path must be /vsp/pas1/pa1.sh
  2. destPath – (Required) Specify the directory path on the Probe where the script must be copied. This directory must be present on the Probe, else the API fails
  3. user – (Optional) Specify the owner of the copied script. If not specified, it Is deployed with the same user running the CPM client
  4. userGroup – (Optional) Specify the user group of the copied script

Request Type: POST 

URL: https://<CMS_IP_Address>/rms/install/script

 

Request JSON:

{

"sourcePath":"/vsp/pas1/pa1.sh",

"destPath":"/opt/vspcpm/script/pas1/pas1.sh",

"user":"virsec",

"userGroup":"virsec",

"probeSet":[

    "PROBE_ID1", 

    "PROBE_ID2"

  ]

}

Sample Response:

{

"status": "SUCCESS",

"message": "Installation of Script started, view status at url - /rms/status/cmd_0c64fb24-0bd8-4d1c-947d-f30483fafd02",

"url": "/rms/status/cmd_0c64fb24-0bd8-4d1c-947d-f30483fafd02",

"api": "install script",

"version": "latest"

}


Error Scenarios

  1. sourcePath
    1. If it is not provided, the API execution fails
    2. It must specify a file inside the LFR. It should not be a directory
  2. destPath – All the parent directories of the destPath must be present on the Probe. If it is not present, the API execution fails
  3. user - Should be an existing user on the Probe. In cases where the user does not exist, the script is copied on the Probe using user running the cpm-client. The change in owner to the specified user fails 
  4. userGroup - Should be an existing user group on the Probes. In cases where the user group does not exist, the script is copied on the Probe using user running the cpm-client. The change in owner to the specified user group fails


VSP-CLI Command

This API is available only for Version 2.11 and Above. It API can be used to execute vsp-cli commands on a Probe machine. 

Accepted Parameters:

  1. arguments– (Required) Accepts all the arguments that are required by vsp-cli command
    1. Example: To execute the command “vsp-cli start ae”, the value for arguments is: [“start”, “ae”]

 Request Type: POST

 URL: https://<CMS_IP_Address>/rms/vspcli

Request JSON:

{

"arguments":["config", "ae", "edit", "logLevel","debug"],

"password":<base64 encoded password>,

"probeSet":[

    "PROBE_ID1", 

    "PROBE_ID2"

  ]

}

Sample Response:

[

  {

    "status": "SUCCESS",

    "api": "vsp-cli command",

    "probeList": [

      {

        "host": "EC2AMAZ-G72FI03",

        "identifier": "439d26d7-694e-4dda-b8b2-d99a569fd743",

        "errorMessage": "",

        "logMessage": "\nEC2AMAZ-G72FI03 ->  Status: SUCCESS  Reason: N/A\nEC2AMAZ-G72FI03 ->\t172.31.28.107\nEC2AMAZ-G72FI03 ->   Status: SUCCESS  Reason: ",

        "status": "SUCCESS"

      }

    ]

  }

]


Error Reponses

Error responses with sample responses applicable for all the APIs mentioned in the previous sections are provided here:

  1. Error Code 400 – Probe ID is invalid or Module ID is invalid
    1. Sample Response 1 – Invalid Probe ID

      {

          "status": "INVALID_PROBEID",

          "errorMessage": "Invalid probeid",

          "apiCommand": "change loglevel to info , for module ae"

      }

    2. Sample Response 2 – Invalid Module ID

      {

          "status": "INVALID_MODULE",

          "errorMessage": "Invalid module, allowed - (ae|fde|fsm|hmm|rmp|vipcs|vma|weba|vmgr|aep)",

          "apiCommand": "change loglevel to info , for module aem"

      }

    3. Sample Response 3 – Invalid SKU value

      {

        "status": "INVALID_PARAMETER",

        "error": "Given sku is invalid. possible values are - (web|host|mem)",

        "api": "install probe"

      }

  2. Error Code 404 – Probe is not found

    {

        "status": "PROBE_NOT_FOUND",

        "errorMessage": "Probe not found - 3c0fae9a-0075-4e8d-8d90-59ad94d0fb13",

        "apiCommand": "change loglevel to info , for module ae"

    }

  3. Error Code 401 – User is unauthorized and user authentication failed



Was this article helpful?