Feature layers in ArcGIS Online (or ArcGIS Enterprise) often contain dynamic records that must be regularly hydrated. There are a number of best practices that you should be aware of to make sure updates are processed in the most efficient manner with minimal impact to downstream users. What is most optimal for you depends on a number of factors including the technical resources you have on hand, the structure and size of the layer and the uptime requirements to your downstream applications and users.
Publish / Overwrite
Let’s start with the most rudimentary method. ArcGIS Pro allows you to republish and overwrite the entire layer using a local feature class. The advantage to this method is that it leverages existing desktop workflows and requires minimal expertise. It offers a quick way to republish an entire layer without relying on truncating and appending features.
This method works for layers that are replaced in their entirety, rather than layers that contain actively edited data or data that must be updated in part. The original feature layer itemId, settings and metadata are left intact. However, there are downsides to using this approach. A small downside to this method is that it will make the feature layer unavailable for a short time.
Caveat: A larger downside is that all of the previous pop-ups, symbology or custom fields (Arcade), you have added to the feature layer will be lost and replaced with whatever is in your source layer(s) in ArcGIS Pro.
You have been warned.
ArcGIS Pro: Append Tool
The Append tool in ArcGIS Pro can be used to add features from a local feature class to an existing feature layer in ArcGIS Online or Enterprise. If you don’t have to add a large amount of records, this tool will work just fine.
Caveat: do not use this tool or the other ArcGIS Pro desktop tools if you are intending to append or edit a large amount of records. Appending thousands or millions of records using this method may introduce problems and lead to substantially long wait times, as the desktop Append tool is not really intended for hosted feature layer updates and does not utilize the high performance Append operation in the ArcGIS REST API.
ArcGIS Online Manual Uploads
Another method at your disposal is the use of manual uploads directly in the ArcGIS Online feature layer item page. Using the Append Data to Layer tool in the Update Data section allows you to upload a csv, excel file, shapefile, geojson or geodatabase containing data to append or update in a feature layer.
When updating existing records or combining updated records with new records (upsert), you have to ensure the unique field values in your upload file match and identify existing records in the layer. If they do not match, the process can fail and lead to lost or corrupted records in your layer.
If you have a single feature layer with multiple layers and tables, you have the ability of choosing to update only a single specific layer in the feature layer.
Using the ArcGIS REST API
If manual updates are not optimal for your organization, there are several options for automating your feature layer updates and appends. The ArcGIS REST API is the primary web API for interacting and managing ArcGIS web services and ArcGIS Online content. The API contains several high performance operations that are specifically tailored for handling feature layers and feature layer content, including the Truncate operation, Append operation and Extract Data utility. Although they perform similar functions to the Append & Truncate tools in ArcGIS Desktop, they are not equivalent. If you need to delete a specific feature or a large group of features, the Delete Features operation is one to use. If you have a large amount of features to delete or are deleting in batches, set the rollbackOnFailure parameter to true to ensure you are not left with partial deletes.
With the REST API, you have the ability to completely replace all features (Truncate and Append Operations), update specific features or add new features, as part of your automated workflow.
Using the ArcGIS API for Python
In addition to the ArcGIS REST API, you can script and automate your feature layer updates by using the ArcGIS API for Python. Under the hood, the Python API leverages the ArcGIS REST API’s high performance operations to update feature layers and interact with ArcGIS Online.
A majority of the functions you will use are in the arcgis.features.FeatureLayer and arcgis.features.Table classes of the Features module. Both the featurelayer and table class have a delete_features and append function. If your hosted feature layer contains several layers and tables, you would have to perform the delete and append functions separately for each sublayer that is a part of the hosted feature layer.
These operations support the same upsert capability as the REST API Append operation, allowing you to update and insert new features simultaneously.
Leveraging Feature Layer Views
One method often used to prevent direct overwrites of layer settings in your downstream maps and apps is to create layer views. Only the feature layer views are then utilized in your maps and applications and your source feature layer is then left un-utilized (not shared). When it is time to update your data, you can actually use the regular Publish / Overwrite tool or the Truncate/Append APIs to update the source feature layer without affecting the specific layer settings (symbology) for your active feature layer views. However, with the full truncate and append operations, data accessibility in the downstream views will still be affected while the source layer is republished.
So which method should you choose?
That depends entirely on the structure of your feature layer, the feature layer settings, the maps and apps utilizing the layer, and your organization’s technical resources and business requirements (layer utilization and uptime). As shown here, when first creating and planning the configuration of your hosted feature layers, future maintenance and data updates should be carefully considered.
