Data Design in ArcGIS Pro

A case study on how to maintain data quality using data design tools.

Design for quality

Data quality is a measure of the condition of data based on a number of factors such as:

  • Accuracy - indicates how correctly the data reflects the real-world conditions it describes.
  • Completeness - reflects how thoroughly the data fulfills an organzation's requirements so it can draw meaningful insights.
  • Consistency - ensures that the data is uniform and reliable and doesn't generate duplicates or conflicts.
  • Validity - indicates if the data that is collected, stored, and formatted is based on an organization's needs.

Data quality is hugely beneficial as it leads to faster and better decision-making across your organization. The more high-quality data you have, the more confident you will be in your decisions. 

In ArcGIS Pro, planning for quality data can be attained using a set of data design tools, each with their own view and ribbon:

Data Design Tools in ArcGIS Pro

Data Design Tools in ArcGIS Pro

This story map will highlight the key aspects of these tools and demonstrate some best practices to achieve the above quality metrics.

Attribute Domains

Use Case 1: You are a GIS Specialist working for White Rock City and have been asked to improve the quality of the city's new light infrastructure data. The goals are to reduce data inconsistency during data entry, help clean up existing data issues, and facilitate the editing experience. To accomplish this goal, you will use attribute domains.

Get more familiar with the Street Lights data and interact with the White Rock City Web Map.


 What are attribute domains? 

 Attribute domains  are a data design feature that describe the available values of a field type. They are used to constrain the values allowed in any attribute for a table or feature class based on a valid list or range of choices.

  • There are two types of domains:
    • Coded domains specify a valid set of values for an attribute. For example, in the diagram on the right, the domain for Luminaire Style has 6 valid values.
    • Range domains specify a range of values by setting minimum and maximum valid values. Eg. A range domain can be set to a Bulb Wattage field to be between 35 and 250 W.
  • Default values simplify data editing processes by automatically inserting the value in a field when a new record is created. 
  • To apply domains and default values to fields, you will use the  Fields view. 

Domain values for the Luminaire Style field

Create attribute domains and the role of default values

The video on the right demonstrates:

  1. Opening the Domains view on the Street Lights layer.
  2. Creating a new coded value domain named Luminaire Style.
  3. Adding domain description text.
  4. Adding coded values and description pairs, then clicking save.
  5. Opening the Fields view.
  6. Assigning the Luminaire Style domain to the field with the same name.
  7. Setting the default value to 'HPS Decorative', as it is the most commonly used.

The editing experience with attribute domains and default values

Data entry video highlights:

  1. Opening the Create Features pane.
  2. Creating a new LED Rope light feature.
  3. Opening the Attributes pane and filling in the appropriate attributes for the newly created feature.
  4. Note that when selecting the Luminaire Style, the HPS styles are listed as valid values, although the light you created has the bulb type LED Rope. You will address this in the next section.

Data verification video highlights:

  1. Selecting a light feature on the map.
  2. In the Attributes pane a warning message pops up if the feature violates the attribute domains.
  3. The error fields are highlighted in red, which allows you to quickly identify the data issue.
  4. To fix it, choose the appropriate attribute from the domain list.
  5. When done, the error message disappears and you can apply your changes.

By creating and applying attribute domains on the Luminaire Style field, you were able to constrain edits to a list of 6 valid values. These valid values reflect the real-world types of street light assets used in White Rock city.

Using domains in this way has improved your data quality by reducing user input mistakes and typos and helps maintain consistency across the street light data.

Subtypes

Use case 2: Each bulb type (HPS, LED, LED with Ropes) has specific luminaire style and wattage characteristics. Having the same default values and attribute domains assigned to all bulbs, at the feature class level, can lead to the incorrect luminaire style being applied to the wrong bulb type. To address this, you will use subtypes to further categorize the Street Lights feature class into three subsets based on the type of bulb.

What are subtypes?

 Subtypes  are a data design element used to create a subset of features in a feature class that share the same attributes. It supports data quality and consistency by allowing data categorization based on those common attributes.

  • Subtypes can only be applied to numeric  field types  (e.g., short, long, double, float).
  • Attribute domains and default values can be applied at the subtype level in a feature class to take full advantage of this data design capability.
Subtypes created for the Bulb Type field and domains applied to it

Create subtypes

The video on the right demonstrates:

  1. Opening the Manage Subtypes dialog to create a new subtype based on the Bulb Type field.
  2. Adding the subtypes code and description pairs.
  3. Opening the Subtypes view from the Street Lights feature class to assess the newly created subtypes.
  4. Assigning domains and default values at the subtypes level, specific for each bulb type.
  5. Opening the Fields view to assess that the Bulb Type field is now highlighted and marked with an asterisk symbol to quickly denote that it has a subtype applied.

Editing experience with subtypes

Data entry video highlights:

  1. Opening the Create Features pane.
  2. Creating a new LED Rope light feature.
  3. Opening the Attributes pane and filling in the appropriate attributes for the newly created feature.
  4. Note that the Luminaire Style value, is already populated with the correct default value, LED Decorative, according to the LED Rope bulb type.

Data editing video highlights:

  1. Changing the subtype of an existing feature without changing the default attributes will generate an error at the domain level and the change cannot be saved to prevent bad data input
  2. Changing the subtype and the default attributes to reflect the new subtype, speeds up the editing process by eliminating manual editing and auto populating the fields.

By applying subtypes, you were able to create dependencies between bulb types and their respective domains and default values. This allows you to further organize the luminaire style, bulb wattage domains, and the default values based on the bulb type. This increases your data accuracy by ensuring that each bulb type is being assigned the right attributes and default values.

Contingent Values

Use case 3: Previously, you applied attribute domains and default values at the subtype level as a solution towards reducing user error during data editing. A great example is the LED Rope subtype. Here, the LED Rope light always has the same light configuration, determined by the bulb type.

Bulb Type: LED Rope => Light Type: Lamp => Pole Style:Decorative => Luminaire Style: LED Decorative => Wattage: 35

To improve the editing experience even further, you will build /expand on the above model and limit the input on a field based on values selected in another field, to sum up the light configuration of a street light feature. This will be extremely helpful for street light features with HPS and LED bulb type, where a unique combination of bulb and pole type further determinates the luminaire style and bulb wattage characteristics. To achieve this, you will create and apply contingent values to the five fields above.

What are contingent values?

 Contingent values  are data design features that allow you to make values in one field dependent on values in another field. They extend the capabilities of attribute domains by making your choice for a value on one field further constrain the domain values that can be placed on another field. 

In the example on the right, the selection of the bulb type as HPS, constrains the Light Style field/domain to four valid options. Then, the unique combination of bulb type HPS, light style Lamp, and pole style Prism, gives the user the remaining valid values for the luminaire style and bulb wattage to complete the light configuration.

This greatly eliminates the risk of user error as it allows a data editor to compose the light configuration of a street light feature (based on selection only) without any manual entry.

  • Attribute domains and Field Groups are key conditions when creating contingent values.
  • Subtypes are not necessary, but work nicely together with contingent values.
  •  Import / export  contingent values tools can be used to automate the creation process.

Contingent values applied to the Bulb Type, Light Style, Pole Style, Luminaire Style, and Bulb Wattage fields.

Create contingent values

The video on the right demonstrates:

  1. Opening the Contingent values view on the Street Lights layer.
  2. Creating a new Field Group named Light Configuration.
  3. Adding the Light Type, Pole Style, Luminaire Style, and Bulb Wattage fields to the group.
  4. Manually adding light configuration contingent values for the HPS subtype.
  5. Using the Import Contingent Values tool to add dependencies for the LED and LED Ropes subtypes.

Editing experience with contingent values

Data entry video highlights:

  1. Opening the Create Features pane.
  2. Creating a new HPS light feature, with the light configuration specific for the Columbia Avenue area.
  3. Opening the Attributes pane and, instead of typing, selecting valid values from the drop down menus in the appropriate light attributes.

Data verification video highlights:

  1. Reviewing LED light features in the Marina area.
  2. The warning message from the Attributes pane let's you quickly spot different errors:
    1. An error caused by a typo is fixed using the contingent values drop down and choosing a valid attribute, specific for the LED configuration.
    2. The Light Type field had an attribute that was not part of the Light type coded domain. Using the contingent value dependencies makes it easy to choose the right light type and match the rest of the light attributes.

Using contingent values, you are able to build on the Bulb Type subtypes and add a deeper level of dependencies between the Light Type, Pole Style, Luminaire Style, and Bulb Wattage fields. This simplifies the editing process for the street light feature configurations specific for HPS and LED by further restricting the valid values that can be applied, and removing any manual entry for populating these fields. This cuts down on user error tremendously.

Attribute Rules

Use case 4: Now that you have a solution to accurately model street light pole configuration, you will help the field workers to better maintain the street light poles. There are four types of tests performed to ensure the quality of the pole structure: the Hammer Test, the Wiring Condition, the Panel Condition, and the Pole Condition. The mean value of these four tests represents the overall Inspection Test Results, which indicates if a pole passed or failed its maintenance verification. To help automate the data editing process and ensure that each light pole passed the inspection test, you will make use of attribute rules.

What are attribute rules?

 Attribute Rules  are a data design feature, complementary to existing rules used in the geodatabase, such as domains, subtypes, and contingent values.  

They enhance the editing experience and improve data quality by automatically populating attributes, restricting invalid edits during edit operations, and performing quality assurance checks on existing features.

The example on the right illustrates how a calculation rule automatically computes the Inspection Test Results, by calculating the mean of the Hammer Test, the Wiring Condition, the Panel Condition, and the Pole Conditions fields.

Then, a validation rule helps the GIS Specialist assess which street light pole features have Inspection Test Results lower than 3 and highlights those as error features that failed the maintenance test.

  • There are three types of attribute rules:
    • Constraint rules: are used to specify and ensure that attribute conditions are met, by returning an error message for rule violations. The rule evaluation occurs immediately during the editing process.
    • Calculation rules: are used to automatically populate attribute values on a feature. The type of rules can be created are: immediate calculation rules, where the rule evaluation occurs immediately during the editing process. Or as batch calculation rules, where the rule evaluation occurs at a user-specific time (on demand).
    • Validation rules: are used to specify and ensure that attribute conditions are met. The rule evaluation occurs for existing features, at a user specific time. If rules are violated, error features are created.

Attribute rules types

  • Immediate calculation and constraint rules are automatically evaluated with edit operations such as:
    • Insert - The rule will be applied when a new feature is added.
    • Update - The rule will be applied when a feature has been updated.
    • Delete - The rule will be applied when a feature has been deleted.
  • Batch calculation and validation rules are evaluated at a user-specified time using the Error Inspector.
  • The  Error Inspector  is used to review and interact with error features.
  • An  Error Layer  stores all error features that violate evaluation of batch calculation and validation rules.
  • The  Arcade  language is used to write attribute rules. Improve your skills in the  Arcade Playground .
  • Enable  Global Id's  on a feature class or table first when creating attribute rules.
  • Enable  Editor Tracking  when working with batch calculation or validation rules.

Calculation and Validation rules explained

Create attribute rules

The video on the right demonstrates:

  1. Opening the Attribute Rules view on the Street Lights layer.
  2. Creating three rules:
    1. Calculate inspection test results - calculation rule that calculates the mean of the four fields and stores the result in the Inspection Test Results field. The rule comes into effect when a new feature is created or an existing one is updated.
    2. Check for null values - constraint rule that ensures the four fields from above are not null. Triggers on insert and update.
    3. Check for light pole features with a failed inspection test - validation rule that returns all the street light poles with test results lower than 3.

Items to note:

  • Each rule is created using an Arcade expression to define the behavior of the rule.

The expression above returns the features with Inspection Results values lower than 3.

  • The calculation and constraint rules are set to trigger when a street light feature is created or updated.
  • For the constraint and validation rules, an error number and message are added. This is extremely helpful as it describes which error has been violated and helps you determine how to fix it.

Editing experience with attribute rules

Data entry video highlights:

  1. Opening the Create Features pane to create a new LED Rope light feature.
  2. The action is blocked by the constraint rule error message which enforces that the Hammer Test, Pole Condition, Wiring Condition, and Panel Condition cannot be empty.
  3. Filling in the fields of interest in the LED Rope template, which satisfies the constraint rule and allows the creation of a new feature.
  4. In the Attributes pane, assessing that the Inspection Result field is already populated with the mean value of the above fields as a result of the calculation rule.

Data verification video highlights:

  1. The constraint warning message is popping up on existing features that violate the constraint rule. This is the benefit of designing this rule to get triggered on an update event.
  2. Filling in the conditions fields automatically triggers the calculation rule to populate the Inspection Result field.
  3. Adding the Error layer, the layer that stores and symbolizes all light features with a failed inspection test.
  4. Opening the Error Inspector view to interact and manage the errors.
  5. Evaluating the validation rule generates three errors in the Error Inspector.
  6. Using the Attributes pane to correct the attributes that are in error.
  7. After completing the cleaning process, a final rule evaluation confirms that no errors remain and all light poles in the field have passed the inspection test.

By adding attribute rules you helped automate the data editing process and facilitate the overall street light pole maintenance process.

  • The constraint rule helped enforce data completeness by ensuring all new and existing light poles have the fields filled with the proper values from the field tests.
  • The calculation rule saved time during editing by automatically calculating the Inspection Results field with the mean value of the other four inspection fields.
  • And the validation rule allowed you to review street light features that failed the inspection test and quickly address and fix them.

Summary

This story map followed a GIS Specialist in their venture to improve the quality of White Rock City’s new street light data. They successfully accomplished this goal by meeting the following data quality objectives:

They achieved data accuracy and data consistency, by creating attribute domains and constraining the luminaire style attributes to a list of valid values to ensure consistency. Then they created subtypes on the Bulb type field and applied the above domains at the feature subtype level. This ensured that each bulb type (HPS, LED, LED with ropes) was being assigned accurate luminaire style attributes. Moreover, they made use of default values to speed up the editing process and eliminate some of the user input.

They ensured data completeness by further enhancing the editing experience for the light infrastructure model. They created contingent values to limit the input of a field based on values selected in other light fields to ensure the light configuration model is complete. Furthermore, the constraint attribute rule enforced data completeness by ensuring all street light features have the Hammer Test, the Wiring Condition, the Panel Condition, and the Pole Condition fields filled in.

They also achieved data validity by creating a calculation attribute rule to automatically compute the mean value of the four test fields and store the valid results in the Inspection Test Result field. Finally, a validation rule helped the GIS Specialist assess the street light features that failed the inspection test and mark them as error features. This further helped them to quickly address and fix the errors.

As mentioned in the introduction, one of the main benefits of having high data quality is that it leads to faster and better decision-making across an organization. In this case study, the GIS Specialist improved the quality of the street light data by using the data design tools in ArcGIS Pro. This enabled them to confidently use the data to assess the current state of the light infrastructure in White Rock City and efficiently utilize city resources by sending field workers to the correct street light poles that failed the inspection test.

Compatibility Considerations

To conclude, the table below sums up the key takeaways and technical characteristics for attribute domains, subtypes, contingent values, and attribute rules. It also includes information about functionality compatibly with other apps across the Esri platform.

*1 In Runtime, attribute rules are supported with web feature layers (Enterprise geodatabase datasets) and mobile geodatabases only. *2 As attribute rules use Arcade as a language to write rules and the Arcade releases are different than the software releases, check out the  Arcade Compatibility Version Matrix  to ensure the successful creation and execution of attribute rules in your workflow.

*1 In Runtime, attribute rules are supported with web feature layers (Enterprise geodatabase datasets) and mobile geodatabases only. *2 As attribute rules use Arcade as a language to write rules and the Arcade releases are different than the software releases, check out the  Arcade Compatibility Version Matrix  to ensure the successful creation and execution of attribute rules in your workflow.

The expression above returns the features with Inspection Results values lower than 3.