AWS plugin

Visit our website to see the data that you can access if you use this plugin to add the data source to SquaredUp:

AWS

Monitor your AWS environment, including EC2, Lambda Functions, CloudWatch and more.

How to add an AWS data source

You will need

The Access key ID and Secret access key of an IAM user with programmatic access.
The AWS Account ID and Target Regions for your AWS resources.

User with Programmatic access

You will need to configure a new or existing user with read-only programmatic access.

See AWS credentials - Programmatic access and Creating an IAM user in your AWS account

Add the AWS data source to SquaredUp

In SquaredUp browse to Data Sources > Add data source and search for the data source. Alternatively, browse to Settings > Data Sources > Add data source.
Enter the Access Key ID and Secret Access Key of the IAM user with programmatic access.
Input the Target Regions for your AWS resources, i.e. us-east-2, eu-west-1
Enter the AWS Account ID
Enter an Account name to help you remember which account and credentials you have used above. For example, you might type prod to remind you that you used the production account details.
Install Sample Dashboards:
Select whether you would like to install sample dashboards with the data source. By default, this is set to on.

Optionally, select whether you would like to restrict access to this data source instance. By default, restricted access is set to off.

Click Test and add to validate the data source configuration.
Testing outcomes
- Testing passed – a success message will be displayed and then the configuration will be saved.
- Testing passed with warnings – warnings will be listed and potential fixes suggested. You can still use the data source with warnings. Select Save with warnings if you believe that you can still use the data source as required with the warnings listed. Alternatively, address the issues listed and then select Rerun tests to validate the data source configuration again. If the validation now passes, click Save.
- Testing Failed – errors will be listed and potential fixes suggested. You cannot use the data source with errors. You are able to select Save with errors if you believe that a system outside of SquaredUp is causing the error that you need to fix. Alternatively, address the issues listed and then select Rerun tests to validate the data source configuration again. If the validation now passes, click Save.
You can edit any data source configurations at any time from Settings > Data Sources.

The index time will depend on the size of your AWS environment.

Using the AWS data streams

Data streams are also installed with the data source see Tips for using the AWS Data Streams

How to create a data stream for this data source:

Create a data stream using the configurable data stream form (simplest option) see Using configurable data streams
Write a custom data stream (advanced use) see Writing a custom data stream (advanced users)

Using configurable data streams

The AWS data source comes with many configurable data streams:

Cost

Explore costs occurred in AWS. Filter by the AWS data source in the tile editor, select Cost, and then click Next

Filters and grouping info at AWS: GetCostAndUsage

Metric: Select the cost metric from Cost Explorer e.g. UnblendedCost.

Granularity: Select how frequently you want to sample the data: Hourly, Daily, or Monthly.

Group By: Add grouping for the cost request by Dimension, Tag or Cost.

Filter: Enter the filter used to remove cost data from the query based on dimensions.

Cost examples:

Group by availability zone

Copy

[
  {
    "Type": "DIMENSION",
    "Key": "AZ"
  }
]

Group by service

Copy

[
  {
    "Type": "DIMENSION",
    "Key": "SERVICE"
  }
]

Filter to lambda cost only

Copy

{
  "Dimensions": {
    "Key": "SERVICE",
    "Values": [
      "AWS Lambda"
    ]
  }
}

Filter to remove tax and developer support

Copy

{
  "Not": {
    "Dimensions": {
      "Key": "SERVICE",
      "Values": [
        "Tax",
        "AWS Support (Developer)"
      ]
    }
  }
}

Tips for using the AWS Data Streams

This data source comes with several data streams that you can use in the tiles on your dashboards. Here you'll find tips and ideas for using these data streams to display the data you're interested in.

Writing a custom data stream (advanced users)

Go to Settings > Data Streams.
Click Add new Data Stream.
Enter a display name for your Data Stream.
Note: The display name is the name that you use to identify your Data Stream in SquaredUp. It has no technical impact and doesn't need to be referenced in the Data Stream's code.
Choose the Data Source this Data Stream is for.
After you've chosen the data source a new field Entry Point appears.
Entry point and code:
What is an entry point?
Each data stream uses an entry point, which can either be global (unscoped) or scoped, and this determines whether the data stream uses the tile scope.
Data streams can be either global or scoped:
- Global data streams are unscoped and return information of a general nature (e.g. "Get the current number of unused hosts").
- A scoped data stream gets information relevant to the specific set objects supplied in the tile scope (e.g. "Get the current session count for these hosts").
To find out which entry point to select and get code examples for the Code field, see the help below.
Click Save to save your Data Stream.

Example for a custom AWS Cost Data Stream

This custom Data Stream will enable you to get different granularity or metrics than the default Cost Data Stream for AWS.

Which entry point do I have to select from the dropdown?

Account Cost

Code example:

Copy

{
    "name": "account-cost",
    "dataSourceConfig": {
        "groupBy": [
            {
                "Type": "DIMENSION",
                "Key": "SERVICE"
            }
        ],
        "metrics": [
            "AmortizedCost"
        ],
        "granularity": "HOURLY",
        "filter": {
            "Dimensions": {
                "Key": "SERVICE",
                "Values": [
                    "AWS Cost Explorer"
                ]
            }
        },
        "rowPath": [
            "ResultsByTime",
            [
                "Groups"
            ]
        ],
        "matches": {
            "sourceType.0": {
                "type": "equals",
                "value": "AWS Account"
            }
        },
        "metadata": [
            {
                "name": "ResultsByTime.TimePeriod.Start",
                "displayName": "Date",
                "shape": "date"
            },
            {
                "name": "ResultsByTime.Groups.Keys.0",
                "displayName": "Service",
                "role": "label",
                "shape": "string"
            },
            {
                "name": "ResultsByTime.Groups.Metrics.AmortizedCost.Amount",
                "displayName": "Cost",
                "role": "value",
                "shape": "currency"
            },
            {
                "name": "ResultsByTime.Groups.Metrics.AmortizedCost.Unit",
                "displayName": "Currency",
                "shape": "string"
            },
            {
                "name": "ResultsByTime.Estimated",
                "displayName": "Estimated",
                "shape": "boolean"
            }
        ]
    }
}

Parameters

name

Mandatory

The internal name of the data stream. Can be used the refer to this data stream in a tile's JSON instead of using the data stream's internal ID.

dataSourceConfig Parameters

Note: You'll find a detailed description about what each parameter does and which values you can use in the AWS API docs: https://docs.aws.amazon.com/aws-cost-management/latest/APIReference/API_GetCostAndUsage.html

`metrics`	Mandatory	The metrics you want to return, for example `AmortizedCost`. If you don't specify `metrics`, the default value is `UnblendedCost`.
`granularity`	Mandatory	The granularity of the data, for example `HOURLY`. If you don't specify `granularity`, the default value is `DAILY`.
`groupBy`	Optional	Use this if you want to group the data. For example, the default AWS Cost Data Stream groups the data by `SERVICE`, which returns the costs for each service in your account. If you don't group by anything, you'll get the costs for your whole account.
`filter`	Optional	Use this if you want to filter AWS costs by different dimensions. For example, if you only want to return the costs of the Cost Explorer service, you can filter to that service here. If you don't use any filter, all cost data for your account will be returned.

matches Parameters

Note: Defining the matches parameter is mandatory.

With the matches parameter you define for which objects the data stream will be shown in SquaredUp. It works like this:
When you configure a tile, you have to choose its scope. If this scope contains objects you specified here in the matches parameter, the data stream will be shown in SquaredUp under Data Streams. If the scope doesn't contain objects specified here, the data stream will be hidden.
This keeps things clean and simple since you'll only see the data stream when it's relevant for the scope you chose. As a best practice you should limit the data stream to objects that make sense for the specific use case of this data stream.

Format for matches:

Copy

//If you want to specify only one value of an object property//
"matches": {
    "ObjectProperty": {
        "type": "equals",
        "value": "ValueOfTheObjectProperty"
    }
},
 
//If you want to specify multiple values for an object property//
"matches": {
    "ObjectProperty": {
         "type": "oneOf",
        "values": ["ValueOfTheObjectProperty1", "ValueOfTheObjectProperty2", "ValueOfTheObjectProperty3"]
    }
},

Example for limiting a data stream to objects:

If you are using multiple values for the object property, you can decide if you want the data stream to be visible for objects that match all of the criteria or at least one of the criteria.

Lets say you have two values you want objects to have in order for the data stream to be visible for them:

a SourceName property with the value AppDynamics (meaning objects that come from the AppDynamics data source)
a type property with the value app (meaning application objects)

If you want the data stream to be visible only for objects that match both of the criteria, your code would look like this:

Copy

  "matches": {
    "sourceName": {
      "type": "equals",
      "value": "AppDynamics"
    },
    "type": {
      "type": "equals",
      "value": "app"
    }
  },

If you want the data stream to be visible for objects that match at least one of the criteria, your code would look like this:

Copy

"matches": [
    {
        "sourceName": {
            "type": "equals",
            "value": "AppDynamics"
        }
    },
    {
        "type": {
            "type": "equals",
            "value": "app"
        }
    }
]

Note: If you run into errors when configuring the matches parameter, check if you're dealing with a global entry point.

Data streams can be either global or scoped:

Global data streams are unscoped and return information of a general nature (e.g. "Get the current number of unused hosts").
A scoped data stream gets information relevant to the specific set objects supplied in the tile scope (e.g. "Get the current session count for these hosts").

rowPath Parameter

Optional

SquaredUp expects data in table form, and here's where you define how the table with your return data will be structured.

The rowPath will tell SquaredUp which items you want to convert into rows.

Example:

Let's say your return data looks like this:

Copy

{
    "generalInfo": "some info",
    "results": [
        {
            "name": "object 1",
            "tags": [
                "tag 1",
                "tag 2",
                "tag 3"
            ]
        },
        {
            "name": "object 2",
            "tags": [
                "tag 1",
                "tag 4"
            ]
        }
    ]
}

Now it depends on what data you want to base your table on, do you want rows per object or per tag?

If you want to see which objects have which tags, your rowPath would be results, and your table would look like this:

`name`	`tags`	`generalInfo`
object 1	tag 1, tag 2, tag 3	some info
object 2	tag 1, tag 4	some info

If you want to turn each tag into a row and see to which objects they are applied, your rowPath would be results.tags, and your table would look like this:

`tags`	`name`	`generalInfo`
tag 1	object 1	some info
tag 1	object 2	some info
tag 2	object 1	some info
tag 3	object 1	some info
tag 4	object 2	some info

As you can see in the example, each parameter gets turned into a column and the items of the parameter you chose as the rowPath will be turned into rows.

You might also have noticed in the example that the parameter generalInfo was included in both cases. This is because after going through all parameters on the same level as your rowPath, SquaredUp will move through all levels above the rowPath and create columns from those parameters, too. Parameters on levels below the rowPath won't be included in the table.

metadata Parameter

Optional, but recommended

The metadata parameters are used to describe columns in order to tell SquaredUp what to do with them. You can do multiple things with the metadata parameters:

Specify how SquaredUp should interpret the columns you return and - to an extent - how their content displayed. You do this by giving each column a shape.
The shape you assign to a column tells SquaredUp what the column contains (for example, a number, a date, a currency, a URL, etc.). Based on the shape SquaredUp decides how to display this column, for example to display a URL as a clickable link.
Filter out or just hide columns.
Only the columns you define in metadata will be returned in the results. This helps you to filter out columns you don't need. If you need the content of a column but don't want to display it, you can use the visible parameter.
Give columns a nicely readable display name.
Assign a specific role to columns
The role you assign to a column tells SquaredUp the purpose of the column. For example, if you have two different columns that contain numbers, you need to assign the role value to the column that contains the actual value you want to use in your visualization.

Note: If you don't specify any metadata, all columns will be returned and SquaredUp will do its best to determine which columns should be used for which purpose. If you're returning pretty simple data, for example just a string and a number, this can work fine. But if you're returning two columns with numbers it gets trickier for SquaredUp to figure out which one is the value and which one is just an ID or some other number.

Parameters:

Tip: Before you start specifying metadata, leave them empty at first and get all the raw data with your new data stream once. In order to do this, finish creating your custom data stream without metadata and create a tile with this data stream. The Table visualization will show you all raw data.
This will give you an overview about all columns and their content and help you decide which columns you need and what their shapes and roles should be. It's also essential for getting the correct column name to reference in the name parameter.
Use this information to go back to the data stream configuration and specifying the metadata.

`name`	Mandatory	Enter the name of the column you are referencing here. To find the name of a column, get the data from this data stream once without any metadata. See the tip above for how to do that. You'll see the column name when you hover over the column in the Table.
`displayName`	Optional	Here you can give the column a user-friendly name
`shape`	Recommended	The shape you assign to a column tells SquaredUp what the column contains (for example, a number, a date, a currency, a URL, etc.). Based on the shape SquaredUp decides how to display this column, for example to display a URL as a clickable link. Note: Please refer to the list of shapes below this table to see available shapes.
`role`	Recommended	The role you assign to a column tells SquaredUp the purpose of the column. For example, if you have two different columns that contain numbers, you need to assign the role `value` to the column that contains the actual value you want to use in your visualization. Note: Please refer to the list of roles below this table to see available roles.
`visible`	Optional	`true` or `false` Use this if you need a columns content but don't need to display the column itself. Example: Column A contains the full link to a ticket in your ticket system. Column B contains the ticket ID. You want to use the ticket ID as a label for the link, turning the long URL into a much nicer to read "Ticket 123". This is why you need the content of column B, to assign it as a label for column A. But since the URL is now displayed as the ticket ID, it would be redundant to still display column B. This is why you hide column B with `false`.

Cloud KB LIVE.10.log-183548538.zip
4 KB Download

Link to workspace	User can link the data source to any workspace they have at least Editor permissions for. Data from the data source can then be viewed by anyone with any access to the workspace. User can share the data source data with anyone they want. User cannot configure the data source in any way, or delete it.
Full Control	User can change the data source configuration, ACL, and delete the data source.

Group by	Date (For each day...)
Aggregate column	Cost (...show me the combined costs...)
Aggregation type	Average or total (... on average or in total.)

`"matches": "none",`	When creating a tile, the Data Stream will be shown as long as no scope is selected. As soon as a scope is selected, the Data Stream will be hidden.
`"matches": "all",`	When creating a tile, the Data Stream will be shown as soon as any scope is selected.

`label`	A column containing user-friendly names. Line Graphs use this role to group data into series. so each label will get its own line in the Line Graph.
`link`	A column containing a link that can be used as a drilldown in Status Blocks.
`timestamp`	A column containing a date to use on the X -axis of a Line Graph.
`unitLabel`	A column containing user-friendly labels for data series, e.g. ‘Duration’. Line Graphs can use this role to label the Y-axis.
`value`	A column containing the numeric value you want to use in your visualization.

AWS plugin

How to add an AWS data source

Using the AWS data streams

Using configurable data streams

Cost examples:

Group by availability zone

Group by service

Filter to lambda cost only

Filter to remove tax and developer support

Tips for using the AWS Data Streams

Writing a custom data stream (advanced users)

Related articles