Enrich Variable Name Lookup

Enrich variable names can be represented using a variety of string representations depending on where you are looking. Frequently there is a need to cross reference between these. The Country.enrich_variables property provides the ability to easily retrieve a Pandas data frame. This data frame can be used to perform cross reference lookups, but there also is a hidden method you can use as well, Country._ba_cntry.get_enrich_variables_from_name_list. This method accepts any iterabe of strings, and will look up the values by checking for a sequence of case insensitive matches in the name, alias, field_name and enrich_field_name columns.

NOTE: This requires at least version 1.9.1 of the Python API. This was included with the Pro 2.9 release, but if you have a previous release of Pro, you can get the most recent release of the Python API by cloning your default Python environment, setting your new environment to be the default and updating the ArcGIS Python API (arcgis) package.

[1]:
from pathlib import Path

from arcgis.geoenrichment import Country
from arcgis.gis import GIS
import pandas as pd

Load Excel Table with Various Representations of Enrich Names

This is a table of name variations for enrich variables I was passed to use as an example of different variations needing to be looked up.

[2]:
dir_prj = Path.cwd().parent
dir_ref = dir_prj/'references'
xlsx_pth = dir_ref/'arcmap_pro_variables.xlsx'

assert xlsx_pth.exists()
[3]:
xlsx_df = pd.read_excel(xlsx_pth)
drop_cols = [c for c in xlsx_df.columns if c.startswith('Unnamed:')]
xlsx_df.drop(columns=drop_cols, inplace=True)
xlsx_df.columns = ['guided_workflow_names', 'pro_enrich_names', 'fieldnames_arcmap', 'fieldnames_pro']

xlsx_df
[3]:
guided_workflow_names pro_enrich_names fieldnames_arcmap fieldnames_pro
0 TOTPOP_CY.2021 Key Demographic Indicators (Esr... populationtotals.totpop_cy TOTPOP_CY populationtotals_totpop_cy
1 HHPOP_CY.2021 Key Demographic Indicators (Esri... householdtotals.tothh_cy HHPOP_CY populationtotals_hhpop_cy
2 FAMPOP_CY.2021 Key Demographic Indicators (Esr... populationtotals.fampop_cy FAMPOP_CY populationtotals_fampop_cy
3 MP14129a_B.2021 Health (Market Potential) (Esr... healthpersonalcare.mp14129a_b MP14129a_B healthpersonalcare_mp14129a_b
4 X11001_X.2021 Education (Consumer Spending) (E... education.x11001_x X11001_X education_x11001_x
5 X11001_A.2021 Education (Consumer Spending) (E... education.x11001_x_a X11001_A education_x11001_x_a
6 X11001_I.2021 Education (Consumer Spending) (E... education.x11001_x_i X11001_I education_x11001_x_i
7 MP09147a_B.2021 Electronics (Market Potential)... electronicsinternet.mp09147a_b MP09147a_B electronicsinternet_mp09147a_b
8 MP09147a_I.2021 Electronics (Market Potential)... electronicsinternet.mp09147a_b_i MP09147a_I electronicsinternet_mp09147a_b_i
9 MP09148a_B.2021 Electronics (Market Potential)... electronicsinternet.mp09148a_b MP09148a_B electronicsinternet_mp09148a_b
10 MP09148a_I.2021 Electronics (Market Potential)... electronicsinternet.mp09148a_b_i MP09148a_I electronicsinternet_mp09148a_b_i
11 MP09134a_B.2021 Electronics (Market Potential)... electronicsinternet.mp09134a_b MP09134a_B electronicsinternet_mp09134a_b
12 MP09134a_I.2021 Electronics (Market Potential)... electronicsinternet.mp09134a_b_i MP09134a_I electronicsinternet_mp09134a_b_i
13 ACSTOTPOP.2015-2019 Key Demographic Indicators... veterans.acstotpop ACSTOTPOP veterans_acstotpop
14 MOETOTPOP.2015-2019 Key Demographic Indicators... veterans.acstotpop_e MOETOTPOP veterans_acstotpop_e
15 RELTOTPOP.2015-2019 Key Demographic Indicators... veterans.acstotpop_r RELTOTPOP veterans_acstotpop_r
16 INDAGRI_CY.2021 Labor Force (Esri) (US 2021 Re... industry.indagri_cy INDAGRI_CY industry_indagri_cy
17 INDMIN_CY.2021 Labor Force (Esri) (US 2021 Ret... industry.indmin_cy INDMIN_CY industry_indmin_cy
18 INDCONS_CY.2021 Labor Force (Esri) (US 2021 Re... industry.indcons_cy INDCONS_CY industry_indcons_cy
19 HHWHTHHR10.2010 Households by Race_Hispanic Or... householdsbyraceofhouseholder.hhwhthhr10 HHWHTHHR10 householdsbyraceofhouseholder_hhwhthhr10
20 HHBLKHHR10.2010 Households by Race_Hispanic Or... householdsbyraceofhouseholder.hhblkhhr10 HHBLKHHR10 householdsbyraceofhouseholder_hhblkhhr10
21 HHAIHHR10.2010 Households by Race_Hispanic Ori... householdsbyraceofhouseholder.hhaihhr10 HHAIHHR10 householdsbyraceofhouseholder_hhaihhr10
22 HHASNHHR10.2010 Households by Race_Hispanic Or... householdsbyraceofhouseholder.hhasnhhr10 HHASNHHR10 householdsbyraceofhouseholder_hhasnhhr10
23 HHPIHHR10.2010 Households by Race_Hispanic Ori... householdsbyraceofhouseholder.hhpihhr10 HHPIHHR10 householdsbyraceofhouseholder_hhpihhr10
24 HHOTHHHR10.2010 Households by Race_Hispanic Or... householdsbyraceofhouseholder.hhothhhr10 HHOTHHHR10 householdsbyraceofhouseholder_hhothhhr10
25 TADULTBASE.2021 Tapestry Market Segmentation (... tapestryadultsnew.tadultbase TADULTBASE tapestryadultsnew_tadultbase
26 TADULT01.2021 Tapestry Market Segmentation (Ad... tapestryadultsnew.tadult01 TADULT01 tapestryadultsnew_tadult01
27 TADULT02.2021 Tapestry Market Segmentation (Ad... tapestryadultsnew.tadult02 TADULT02 tapestryadultsnew_tadult02
28 POPGRW10CY.2021 Key Demographic Indicators (Es... populationtotals.popgrw10cy POPGRW10CY populationtotals_popgrw10cy
29 DIVINDX_CY.2021 Race and Hispanic Origin (Esri... raceandhispanicorigin.divindx_cy DIVINDX_CY raceandhispanicorigin_divindx_cy
30 MEDHINC_CY.2021 Income (Esri) (Esri USA 2021) householdincome.medhinc_cy MEDHINC_CY householdincome_medhinc_cy
31 ACSMEDHINC.2015-2019 Income (ACS) (Esri USA 20... householdincome.acsmedhinc ACSMEDHINC householdincome_acsmedhinc
32 MOEMEDHINC.2015-2019 Income (ACS) (Esri USA 20... householdincome.acsmedhinc_e MOEMEDHINC householdincome_acsmedhinc_e
33 RELMEDHINC.2015-2019 Income (ACS) (Esri USA 20... householdincome.acsmedhinc_r RELMEDHINC householdincome_acsmedhinc_r

Create a Country Object Instance

Create a Country object instance referencing the country you are looking for variables in.

[4]:
usa = Country('usa')

Perform Lookups

Now, we just need to provide an interable of strings to be matched into the get_enrich_variables_from_list function.

ArcGIS Pro Enrich Names

If using a list of names used to enrich with ArcGIS Pro, you can easily look up these values.

[5]:
print(list(xlsx_df.pro_enrich_names))
['populationtotals.totpop_cy', 'householdtotals.tothh_cy', 'populationtotals.fampop_cy', 'healthpersonalcare.mp14129a_b', 'education.x11001_x', 'education.x11001_x_a', 'education.x11001_x_i', 'electronicsinternet.mp09147a_b', 'electronicsinternet.mp09147a_b_i', 'electronicsinternet.mp09148a_b', 'electronicsinternet.mp09148a_b_i', 'electronicsinternet.mp09134a_b', 'electronicsinternet.mp09134a_b_i', 'veterans.acstotpop', 'veterans.acstotpop_e', 'veterans.acstotpop_r', 'industry.indagri_cy', 'industry.indmin_cy', 'industry.indcons_cy', 'householdsbyraceofhouseholder.hhwhthhr10', 'householdsbyraceofhouseholder.hhblkhhr10', 'householdsbyraceofhouseholder.hhaihhr10', 'householdsbyraceofhouseholder.hhasnhhr10', 'householdsbyraceofhouseholder.hhpihhr10', 'householdsbyraceofhouseholder.hhothhhr10', 'tapestryadultsnew.tadultbase', 'tapestryadultsnew.tadult01', 'tapestryadultsnew.tadult02', 'populationtotals.popgrw10cy', 'raceandhispanicorigin.divindx_cy', 'householdincome.medhinc_cy', 'householdincome.acsmedhinc', 'householdincome.acsmedhinc_e', 'householdincome.acsmedhinc_r']
[6]:
usa._ba_cntry.get_enrich_variables_from_iterable(xlsx_df.pro_enrich_names)
[6]:
name alias data_collection enrich_name enrich_field_name
0 ACSMEDHINC 2019 Median HH Income (ACS 5-Yr) householdincome householdincome.ACSMEDHINC householdincome_ACSMEDHINC
1 MEDHINC_CY 2021 Median Household Income householdincome householdincome.MEDHINC_CY householdincome_MEDHINC_CY
2 MP09147a_B 2021 Own Any Tablet ElectronicsInternet ElectronicsInternet.MP09147a_B ElectronicsInternet_MP09147a_B
3 MP09148a_B 2021 Own Any E-Reader ElectronicsInternet ElectronicsInternet.MP09148a_B ElectronicsInternet_MP09148a_B
4 MP09134a_B 2021 Own E-Reader/Tablet: Amazon Kindle ElectronicsInternet ElectronicsInternet.MP09134a_B ElectronicsInternet_MP09134a_B
5 X11001_X 2021 Education education education.X11001_X education_X11001_X
6 MP14129a_B 2021 Typically Spend 1-3 Hrs Exercising/Wk HealthPersonalCare HealthPersonalCare.MP14129a_B HealthPersonalCare_MP14129a_B
7 INDAGRI_CY 2021 Industry: Agriculture industry industry.INDAGRI_CY industry_INDAGRI_CY
8 INDMIN_CY 2021 Industry: Mining industry industry.INDMIN_CY industry_INDMIN_CY
9 INDCONS_CY 2021 Industry: Construction industry industry.INDCONS_CY industry_INDCONS_CY
10 HHWHTHHR10 2010 HHs w/White HHr householdsbyraceofhouseholder householdsbyraceofhouseholder.HHWHTHHR10 householdsbyraceofhouseholder_HHWHTHHR10
11 HHBLKHHR10 2010 HHs w/Black HHr householdsbyraceofhouseholder householdsbyraceofhouseholder.HHBLKHHR10 householdsbyraceofhouseholder_HHBLKHHR10
12 HHAIHHR10 2010 HHs w/Amer Indian HHr householdsbyraceofhouseholder householdsbyraceofhouseholder.HHAIHHR10 householdsbyraceofhouseholder_HHAIHHR10
13 HHASNHHR10 2010 HHs w/Asian HHr householdsbyraceofhouseholder householdsbyraceofhouseholder.HHASNHHR10 householdsbyraceofhouseholder_HHASNHHR10
14 HHPIHHR10 2010 HHs w/Pacific Isl HHr householdsbyraceofhouseholder householdsbyraceofhouseholder.HHPIHHR10 householdsbyraceofhouseholder_HHPIHHR10
15 HHOTHHHR10 2010 HHs w/Other Race HHr householdsbyraceofhouseholder householdsbyraceofhouseholder.HHOTHHHR10 householdsbyraceofhouseholder_HHOTHHHR10
16 TOTHH_CY 2021 Total Households householdtotals householdtotals.TOTHH_CY householdtotals_TOTHH_CY
17 TADULTBASE 2021 Tapestry Adult Pop Base tapestryadultsNEW tapestryadultsNEW.TADULTBASE tapestryadultsNEW_TADULTBASE
18 TADULT01 2021 Pop 18+ in Tapestry Seg 1A tapestryadultsNEW tapestryadultsNEW.TADULT01 tapestryadultsNEW_TADULT01
19 TADULT02 2021 Pop 18+ in Tapestry Seg 1B tapestryadultsNEW tapestryadultsNEW.TADULT02 tapestryadultsNEW_TADULT02
20 TOTPOP_CY 2021 Total Population populationtotals populationtotals.TOTPOP_CY populationtotals_TOTPOP_CY
21 FAMPOP_CY 2021 Family Population populationtotals populationtotals.FAMPOP_CY populationtotals_FAMPOP_CY
22 POPGRW10CY 2010-2021 Growth Rate: Population populationtotals populationtotals.POPGRW10CY populationtotals_POPGRW10CY
23 DIVINDX_CY 2021 Diversity Index raceandhispanicorigin raceandhispanicorigin.DIVINDX_CY raceandhispanicorigin_DIVINDX_CY
24 ACSTOTPOP 2019 Total Population (ACS 5-Yr) veterans veterans.ACSTOTPOP veterans_ACSTOTPOP

ArcMap Enrich Names

ArcMap required a slightly different string format. Hence, if migrating from ArcMap, the same workflow can be used to retrieve these values as well.

[7]:
print(list(xlsx_df.fieldnames_arcmap))
['TOTPOP_CY', 'HHPOP_CY', 'FAMPOP_CY', 'MP14129a_B', 'X11001_X', 'X11001_A', 'X11001_I', 'MP09147a_B', 'MP09147a_I', 'MP09148a_B', 'MP09148a_I', 'MP09134a_B', 'MP09134a_I', 'ACSTOTPOP', 'MOETOTPOP', 'RELTOTPOP', 'INDAGRI_CY', 'INDMIN_CY', 'INDCONS_CY', 'HHWHTHHR10', 'HHBLKHHR10', 'HHAIHHR10', 'HHASNHHR10', 'HHPIHHR10', 'HHOTHHHR10', 'TADULTBASE', 'TADULT01', 'TADULT02', 'POPGRW10CY', 'DIVINDX_CY', 'MEDHINC_CY', 'ACSMEDHINC', 'MOEMEDHINC', 'RELMEDHINC']
[8]:
usa._ba_cntry.get_enrich_variables_from_iterable(xlsx_df.fieldnames_arcmap)
[8]:
name alias data_collection enrich_name enrich_field_name
0 MEDHINC_CY 2021 Median Household Income Health Health.MEDHINC_CY Health_MEDHINC_CY
1 MOETOTPOP 2019 Total Population MOE (ACS 5-Yr) ACS_Housing_Summary_rep ACS_Housing_Summary_rep.MOETOTPOP ACS_Housing_Summary_rep_MOETOTPOP
2 TADULTBASE 2021 Tapestry Adult Pop Base travelMPI travelMPI.TADULTBASE travelMPI_TADULTBASE
3 TOTPOP_CY 2021 Total Population Age_by_Sex_by_Race_Profile_rep Age_by_Sex_by_Race_Profile_rep.TOTPOP_CY Age_by_Sex_by_Race_Profile_rep_TOTPOP_CY
4 DIVINDX_CY 2021 Diversity Index Policy Policy.DIVINDX_CY Policy_DIVINDX_CY
5 ACSMEDHINC 2019 Median HH Income (ACS 5-Yr) householdincome householdincome.ACSMEDHINC householdincome_ACSMEDHINC
6 MOEMEDHINC 2019 Median HH Income MOE (ACS 5-Yr) householdincome householdincome.MOEMEDHINC householdincome_MOEMEDHINC
7 RELMEDHINC 2019 Median HH Income REL (ACS 5-Yr) householdincome householdincome.RELMEDHINC householdincome_RELMEDHINC
8 X11001_I 2021 Index: Education Community_Profile_rep Community_Profile_rep.X11001_I Community_Profile_rep_X11001_I
9 MP09147a_B 2021 Own Any Tablet ElectronicsInternet ElectronicsInternet.MP09147a_B ElectronicsInternet_MP09147a_B
10 MP09147a_I 2021 Index: Own Any Tablet ElectronicsInternet ElectronicsInternet.MP09147a_I ElectronicsInternet_MP09147a_I
11 MP09148a_B 2021 Own Any E-Reader ElectronicsInternet ElectronicsInternet.MP09148a_B ElectronicsInternet_MP09148a_B
12 MP09148a_I 2021 Index: Own Any E-Reader ElectronicsInternet ElectronicsInternet.MP09148a_I ElectronicsInternet_MP09148a_I
13 MP09134a_B 2021 Own E-Reader/Tablet: Amazon Kindle ElectronicsInternet ElectronicsInternet.MP09134a_B ElectronicsInternet_MP09134a_B
14 MP09134a_I 2021 Index: Own E-Reader/Tablet: Amazon Kindle ElectronicsInternet ElectronicsInternet.MP09134a_I ElectronicsInternet_MP09134a_I
15 X11001_X 2021 Education education education.X11001_X education_X11001_X
16 X11001_A 2021 Avg: Education education education.X11001_A education_X11001_A
17 MP14129a_B 2021 Typically Spend 1-3 Hrs Exercising/Wk HealthPersonalCare HealthPersonalCare.MP14129a_B HealthPersonalCare_MP14129a_B
18 INDAGRI_CY 2021 Industry: Agriculture industry industry.INDAGRI_CY industry_INDAGRI_CY
19 INDMIN_CY 2021 Industry: Mining industry industry.INDMIN_CY industry_INDMIN_CY
20 INDCONS_CY 2021 Industry: Construction industry industry.INDCONS_CY industry_INDCONS_CY
21 HHWHTHHR10 2010 HHs w/White HHr householdsbyraceofhouseholder householdsbyraceofhouseholder.HHWHTHHR10 householdsbyraceofhouseholder_HHWHTHHR10
22 HHBLKHHR10 2010 HHs w/Black HHr householdsbyraceofhouseholder householdsbyraceofhouseholder.HHBLKHHR10 householdsbyraceofhouseholder_HHBLKHHR10
23 HHAIHHR10 2010 HHs w/Amer Indian HHr householdsbyraceofhouseholder householdsbyraceofhouseholder.HHAIHHR10 householdsbyraceofhouseholder_HHAIHHR10
24 HHASNHHR10 2010 HHs w/Asian HHr householdsbyraceofhouseholder householdsbyraceofhouseholder.HHASNHHR10 householdsbyraceofhouseholder_HHASNHHR10
25 HHPIHHR10 2010 HHs w/Pacific Isl HHr householdsbyraceofhouseholder householdsbyraceofhouseholder.HHPIHHR10 householdsbyraceofhouseholder_HHPIHHR10
26 HHOTHHHR10 2010 HHs w/Other Race HHr householdsbyraceofhouseholder householdsbyraceofhouseholder.HHOTHHHR10 householdsbyraceofhouseholder_HHOTHHHR10
27 POPGRW10CY 2010-2021 Growth Rate: Population KeyUSFacts KeyUSFacts.POPGRW10CY KeyUSFacts_POPGRW10CY
28 TADULT01 2021 Pop 18+ in Tapestry Seg 1A tapestryadultsNEW tapestryadultsNEW.TADULT01 tapestryadultsNEW_TADULT01
29 TADULT02 2021 Pop 18+ in Tapestry Seg 1B tapestryadultsNEW tapestryadultsNEW.TADULT02 tapestryadultsNEW_TADULT02
30 ACSTOTPOP 2019 Total Population (ACS 5-Yr) population population.ACSTOTPOP population_ACSTOTPOP
31 RELTOTPOP 2019 Total Population REL (ACS 5-Yr) population population.RELTOTPOP population_RELTOTPOP
32 HHPOP_CY 2021 Household Population populationtotals populationtotals.HHPOP_CY populationtotals_HHPOP_CY
33 FAMPOP_CY 2021 Family Population populationtotals populationtotals.FAMPOP_CY populationtotals_FAMPOP_CY

Field Names

Once enriched, the enrich_name string has the period replaced for the output attibute field name (enrich_field_name). We can use also use these output field name values to lookup relevant rows in the enrich_variables table.

[9]:
print(list(xlsx_df.fieldnames_pro))
['populationtotals_totpop_cy', 'populationtotals_hhpop_cy', 'populationtotals_fampop_cy', 'healthpersonalcare_mp14129a_b', 'education_x11001_x', 'education_x11001_x_a', 'education_x11001_x_i', 'electronicsinternet_mp09147a_b', 'electronicsinternet_mp09147a_b_i', 'electronicsinternet_mp09148a_b', 'electronicsinternet_mp09148a_b_i', 'electronicsinternet_mp09134a_b', 'electronicsinternet_mp09134a_b_i', 'veterans_acstotpop', 'veterans_acstotpop_e', 'veterans_acstotpop_r', 'industry_indagri_cy', 'industry_indmin_cy', 'industry_indcons_cy', 'householdsbyraceofhouseholder_hhwhthhr10', 'householdsbyraceofhouseholder_hhblkhhr10', 'householdsbyraceofhouseholder_hhaihhr10', 'householdsbyraceofhouseholder_hhasnhhr10', 'householdsbyraceofhouseholder_hhpihhr10', 'householdsbyraceofhouseholder_hhothhhr10', 'tapestryadultsnew_tadultbase', 'tapestryadultsnew_tadult01', 'tapestryadultsnew_tadult02', 'populationtotals_popgrw10cy', 'raceandhispanicorigin_divindx_cy', 'householdincome_medhinc_cy', 'householdincome_acsmedhinc', 'householdincome_acsmedhinc_e', 'householdincome_acsmedhinc_r']
[10]:
usa._ba_cntry.get_enrich_variables_from_iterable(xlsx_df.fieldnames_pro)
[10]:
name alias data_collection enrich_name enrich_field_name
0 ACSMEDHINC 2019 Median HH Income (ACS 5-Yr) householdincome householdincome.ACSMEDHINC householdincome_ACSMEDHINC
1 MEDHINC_CY 2021 Median Household Income householdincome householdincome.MEDHINC_CY householdincome_MEDHINC_CY
2 MP09147a_B 2021 Own Any Tablet ElectronicsInternet ElectronicsInternet.MP09147a_B ElectronicsInternet_MP09147a_B
3 MP09148a_B 2021 Own Any E-Reader ElectronicsInternet ElectronicsInternet.MP09148a_B ElectronicsInternet_MP09148a_B
4 MP09134a_B 2021 Own E-Reader/Tablet: Amazon Kindle ElectronicsInternet ElectronicsInternet.MP09134a_B ElectronicsInternet_MP09134a_B
5 X11001_X 2021 Education education education.X11001_X education_X11001_X
6 MP14129a_B 2021 Typically Spend 1-3 Hrs Exercising/Wk HealthPersonalCare HealthPersonalCare.MP14129a_B HealthPersonalCare_MP14129a_B
7 INDAGRI_CY 2021 Industry: Agriculture industry industry.INDAGRI_CY industry_INDAGRI_CY
8 INDMIN_CY 2021 Industry: Mining industry industry.INDMIN_CY industry_INDMIN_CY
9 INDCONS_CY 2021 Industry: Construction industry industry.INDCONS_CY industry_INDCONS_CY
10 HHWHTHHR10 2010 HHs w/White HHr householdsbyraceofhouseholder householdsbyraceofhouseholder.HHWHTHHR10 householdsbyraceofhouseholder_HHWHTHHR10
11 HHBLKHHR10 2010 HHs w/Black HHr householdsbyraceofhouseholder householdsbyraceofhouseholder.HHBLKHHR10 householdsbyraceofhouseholder_HHBLKHHR10
12 HHAIHHR10 2010 HHs w/Amer Indian HHr householdsbyraceofhouseholder householdsbyraceofhouseholder.HHAIHHR10 householdsbyraceofhouseholder_HHAIHHR10
13 HHASNHHR10 2010 HHs w/Asian HHr householdsbyraceofhouseholder householdsbyraceofhouseholder.HHASNHHR10 householdsbyraceofhouseholder_HHASNHHR10
14 HHPIHHR10 2010 HHs w/Pacific Isl HHr householdsbyraceofhouseholder householdsbyraceofhouseholder.HHPIHHR10 householdsbyraceofhouseholder_HHPIHHR10
15 HHOTHHHR10 2010 HHs w/Other Race HHr householdsbyraceofhouseholder householdsbyraceofhouseholder.HHOTHHHR10 householdsbyraceofhouseholder_HHOTHHHR10
16 TADULTBASE 2021 Tapestry Adult Pop Base tapestryadultsNEW tapestryadultsNEW.TADULTBASE tapestryadultsNEW_TADULTBASE
17 TADULT01 2021 Pop 18+ in Tapestry Seg 1A tapestryadultsNEW tapestryadultsNEW.TADULT01 tapestryadultsNEW_TADULT01
18 TADULT02 2021 Pop 18+ in Tapestry Seg 1B tapestryadultsNEW tapestryadultsNEW.TADULT02 tapestryadultsNEW_TADULT02
19 TOTPOP_CY 2021 Total Population populationtotals populationtotals.TOTPOP_CY populationtotals_TOTPOP_CY
20 HHPOP_CY 2021 Household Population populationtotals populationtotals.HHPOP_CY populationtotals_HHPOP_CY
21 FAMPOP_CY 2021 Family Population populationtotals populationtotals.FAMPOP_CY populationtotals_FAMPOP_CY
22 POPGRW10CY 2010-2021 Growth Rate: Population populationtotals populationtotals.POPGRW10CY populationtotals_POPGRW10CY
23 DIVINDX_CY 2021 Diversity Index raceandhispanicorigin raceandhispanicorigin.DIVINDX_CY raceandhispanicorigin_DIVINDX_CY
24 ACSTOTPOP 2019 Total Population (ACS 5-Yr) veterans veterans.ACSTOTPOP veterans_ACSTOTPOP

NOTE: Another way to get this list is simply using a list comprehension listing the field names using arcpy.ListFields.

import arcpy
from arcgis.gis import Country
pth_to_fc = r'C:/to/somwhere/somedb.fdgb/fc_of_stuff'
field_name_list = [f.name for f in arcpy.ListFields(pth_to_fc)]
usa = Country('usa')
usa._ba_cntry.get_enrich_variables_from_name_list(field_name_list)

Preprocessing with Lookup

The first column we are saving for last. These values do not have explicit matches. However, using pandas, we can split the string at the period and only keep the first part. This first part can then be looked up since it matches the name column.

[11]:
xlsx_df.guided_workflow_names.head()
[11]:
0    TOTPOP_CY.2021 Key Demographic Indicators (Esr...
1    HHPOP_CY.2021 Key Demographic Indicators (Esri...
2    FAMPOP_CY.2021 Key Demographic Indicators (Esr...
3    MP14129a_B.2021 Health (Market Potential) (Esr...
4    X11001_X.2021 Education (Consumer Spending) (E...
Name: guided_workflow_names, dtype: object

We do it slightly differently here. Instead of simply creating an iterable to use for lookup, in this case, we add the search_names to the existing dataframe generated from the Excel table we read in.

[12]:
xlsx_df['search_names'] = xlsx_df.guided_workflow_names.str.split('.').apply(lambda val: val[0])

xlsx_df.head()
[12]:
guided_workflow_names pro_enrich_names fieldnames_arcmap fieldnames_pro search_names
0 TOTPOP_CY.2021 Key Demographic Indicators (Esr... populationtotals.totpop_cy TOTPOP_CY populationtotals_totpop_cy TOTPOP_CY
1 HHPOP_CY.2021 Key Demographic Indicators (Esri... householdtotals.tothh_cy HHPOP_CY populationtotals_hhpop_cy HHPOP_CY
2 FAMPOP_CY.2021 Key Demographic Indicators (Esr... populationtotals.fampop_cy FAMPOP_CY populationtotals_fampop_cy FAMPOP_CY
3 MP14129a_B.2021 Health (Market Potential) (Esr... healthpersonalcare.mp14129a_b MP14129a_B healthpersonalcare_mp14129a_b MP14129a_B
4 X11001_X.2021 Education (Consumer Spending) (E... education.x11001_x X11001_X education_x11001_x X11001_X

Since the function just needs an Iterable object, we can use the column (Pandas Series) we just created to look up valid values.

[13]:
enrich_vars = usa._ba_cntry.get_enrich_variables_from_iterable(xlsx_df.search_names)

enrich_vars
[13]:
name alias data_collection enrich_name enrich_field_name
0 MEDHINC_CY 2021 Median Household Income Health Health.MEDHINC_CY Health_MEDHINC_CY
1 MOETOTPOP 2019 Total Population MOE (ACS 5-Yr) ACS_Housing_Summary_rep ACS_Housing_Summary_rep.MOETOTPOP ACS_Housing_Summary_rep_MOETOTPOP
2 TADULTBASE 2021 Tapestry Adult Pop Base travelMPI travelMPI.TADULTBASE travelMPI_TADULTBASE
3 TOTPOP_CY 2021 Total Population Age_by_Sex_by_Race_Profile_rep Age_by_Sex_by_Race_Profile_rep.TOTPOP_CY Age_by_Sex_by_Race_Profile_rep_TOTPOP_CY
4 DIVINDX_CY 2021 Diversity Index Policy Policy.DIVINDX_CY Policy_DIVINDX_CY
5 ACSMEDHINC 2019 Median HH Income (ACS 5-Yr) householdincome householdincome.ACSMEDHINC householdincome_ACSMEDHINC
6 MOEMEDHINC 2019 Median HH Income MOE (ACS 5-Yr) householdincome householdincome.MOEMEDHINC householdincome_MOEMEDHINC
7 RELMEDHINC 2019 Median HH Income REL (ACS 5-Yr) householdincome householdincome.RELMEDHINC householdincome_RELMEDHINC
8 X11001_I 2021 Index: Education Community_Profile_rep Community_Profile_rep.X11001_I Community_Profile_rep_X11001_I
9 MP09147a_B 2021 Own Any Tablet ElectronicsInternet ElectronicsInternet.MP09147a_B ElectronicsInternet_MP09147a_B
10 MP09147a_I 2021 Index: Own Any Tablet ElectronicsInternet ElectronicsInternet.MP09147a_I ElectronicsInternet_MP09147a_I
11 MP09148a_B 2021 Own Any E-Reader ElectronicsInternet ElectronicsInternet.MP09148a_B ElectronicsInternet_MP09148a_B
12 MP09148a_I 2021 Index: Own Any E-Reader ElectronicsInternet ElectronicsInternet.MP09148a_I ElectronicsInternet_MP09148a_I
13 MP09134a_B 2021 Own E-Reader/Tablet: Amazon Kindle ElectronicsInternet ElectronicsInternet.MP09134a_B ElectronicsInternet_MP09134a_B
14 MP09134a_I 2021 Index: Own E-Reader/Tablet: Amazon Kindle ElectronicsInternet ElectronicsInternet.MP09134a_I ElectronicsInternet_MP09134a_I
15 X11001_X 2021 Education education education.X11001_X education_X11001_X
16 X11001_A 2021 Avg: Education education education.X11001_A education_X11001_A
17 MP14129a_B 2021 Typically Spend 1-3 Hrs Exercising/Wk HealthPersonalCare HealthPersonalCare.MP14129a_B HealthPersonalCare_MP14129a_B
18 INDAGRI_CY 2021 Industry: Agriculture industry industry.INDAGRI_CY industry_INDAGRI_CY
19 INDMIN_CY 2021 Industry: Mining industry industry.INDMIN_CY industry_INDMIN_CY
20 INDCONS_CY 2021 Industry: Construction industry industry.INDCONS_CY industry_INDCONS_CY
21 HHWHTHHR10 2010 HHs w/White HHr householdsbyraceofhouseholder householdsbyraceofhouseholder.HHWHTHHR10 householdsbyraceofhouseholder_HHWHTHHR10
22 HHBLKHHR10 2010 HHs w/Black HHr householdsbyraceofhouseholder householdsbyraceofhouseholder.HHBLKHHR10 householdsbyraceofhouseholder_HHBLKHHR10
23 HHAIHHR10 2010 HHs w/Amer Indian HHr householdsbyraceofhouseholder householdsbyraceofhouseholder.HHAIHHR10 householdsbyraceofhouseholder_HHAIHHR10
24 HHASNHHR10 2010 HHs w/Asian HHr householdsbyraceofhouseholder householdsbyraceofhouseholder.HHASNHHR10 householdsbyraceofhouseholder_HHASNHHR10
25 HHPIHHR10 2010 HHs w/Pacific Isl HHr householdsbyraceofhouseholder householdsbyraceofhouseholder.HHPIHHR10 householdsbyraceofhouseholder_HHPIHHR10
26 HHOTHHHR10 2010 HHs w/Other Race HHr householdsbyraceofhouseholder householdsbyraceofhouseholder.HHOTHHHR10 householdsbyraceofhouseholder_HHOTHHHR10
27 POPGRW10CY 2010-2021 Growth Rate: Population KeyUSFacts KeyUSFacts.POPGRW10CY KeyUSFacts_POPGRW10CY
28 TADULT01 2021 Pop 18+ in Tapestry Seg 1A tapestryadultsNEW tapestryadultsNEW.TADULT01 tapestryadultsNEW_TADULT01
29 TADULT02 2021 Pop 18+ in Tapestry Seg 1B tapestryadultsNEW tapestryadultsNEW.TADULT02 tapestryadultsNEW_TADULT02
30 ACSTOTPOP 2019 Total Population (ACS 5-Yr) population population.ACSTOTPOP population_ACSTOTPOP
31 RELTOTPOP 2019 Total Population REL (ACS 5-Yr) population population.RELTOTPOP population_RELTOTPOP
32 HHPOP_CY 2021 Household Population populationtotals populationtotals.HHPOP_CY populationtotals_HHPOP_CY
33 FAMPOP_CY 2021 Family Population populationtotals populationtotals.FAMPOP_CY populationtotals_FAMPOP_CY

Concantenating for Local Enrich

The arcpy.ba.EnrichLayer tool in ArcGIS Pro needs a semicolon separated string of enrich_name values for input. From any of the above results, this can quickly be created from the enich_name column.

[14]:
vars_str = ';'.join(enrich_vars.enrich_name.values)

print(vars_str)
Health.MEDHINC_CY;ACS_Housing_Summary_rep.MOETOTPOP;travelMPI.TADULTBASE;Age_by_Sex_by_Race_Profile_rep.TOTPOP_CY;Policy.DIVINDX_CY;householdincome.ACSMEDHINC;householdincome.MOEMEDHINC;householdincome.RELMEDHINC;Community_Profile_rep.X11001_I;ElectronicsInternet.MP09147a_B;ElectronicsInternet.MP09147a_I;ElectronicsInternet.MP09148a_B;ElectronicsInternet.MP09148a_I;ElectronicsInternet.MP09134a_B;ElectronicsInternet.MP09134a_I;education.X11001_X;education.X11001_A;HealthPersonalCare.MP14129a_B;industry.INDAGRI_CY;industry.INDMIN_CY;industry.INDCONS_CY;householdsbyraceofhouseholder.HHWHTHHR10;householdsbyraceofhouseholder.HHBLKHHR10;householdsbyraceofhouseholder.HHAIHHR10;householdsbyraceofhouseholder.HHASNHHR10;householdsbyraceofhouseholder.HHPIHHR10;householdsbyraceofhouseholder.HHOTHHHR10;KeyUSFacts.POPGRW10CY;tapestryadultsNEW.TADULT01;tapestryadultsNEW.TADULT02;population.ACSTOTPOP;population.RELTOTPOP;populationtotals.HHPOP_CY;populationtotals.FAMPOP_CY

Adding Onto Existing Table

Also, if we want, we can add the values needed for the Enrich Layer tool onto our existing table.

[15]:
add_df = enrich_vars[['name', 'enrich_name']].set_index('name')

add_df.head()
[15]:
enrich_name
name
MEDHINC_CY Health.MEDHINC_CY
MOETOTPOP ACS_Housing_Summary_rep.MOETOTPOP
TADULTBASE travelMPI.TADULTBASE
TOTPOP_CY Age_by_Sex_by_Race_Profile_rep.TOTPOP_CY
DIVINDX_CY Policy.DIVINDX_CY
[16]:
joined_df = xlsx_df.join(add_df, on='search_names')

joined_df.info()
joined_df.head()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 34 entries, 0 to 33
Data columns (total 6 columns):
 #   Column                 Non-Null Count  Dtype
---  ------                 --------------  -----
 0   guided_workflow_names  34 non-null     object
 1   pro_enrich_names       34 non-null     object
 2   fieldnames_arcmap      34 non-null     object
 3   fieldnames_pro         34 non-null     object
 4   search_names           34 non-null     object
 5   enrich_name            34 non-null     object
dtypes: object(6)
memory usage: 1.7+ KB
[16]:
guided_workflow_names pro_enrich_names fieldnames_arcmap fieldnames_pro search_names enrich_name
0 TOTPOP_CY.2021 Key Demographic Indicators (Esr... populationtotals.totpop_cy TOTPOP_CY populationtotals_totpop_cy TOTPOP_CY Age_by_Sex_by_Race_Profile_rep.TOTPOP_CY
1 HHPOP_CY.2021 Key Demographic Indicators (Esri... householdtotals.tothh_cy HHPOP_CY populationtotals_hhpop_cy HHPOP_CY populationtotals.HHPOP_CY
2 FAMPOP_CY.2021 Key Demographic Indicators (Esr... populationtotals.fampop_cy FAMPOP_CY populationtotals_fampop_cy FAMPOP_CY populationtotals.FAMPOP_CY
3 MP14129a_B.2021 Health (Market Potential) (Esr... healthpersonalcare.mp14129a_b MP14129a_B healthpersonalcare_mp14129a_b MP14129a_B HealthPersonalCare.MP14129a_B
4 X11001_X.2021 Education (Consumer Spending) (E... education.x11001_x X11001_X education_x11001_x X11001_X education.X11001_X

Finally, if all we want is the input values, and the enrich names, we can quickly filter to just these columns.

[17]:
vals_df = joined_df.loc[:,['guided_workflow_names', 'enrich_name']]

vals_df
[17]:
guided_workflow_names enrich_name
0 TOTPOP_CY.2021 Key Demographic Indicators (Esr... Age_by_Sex_by_Race_Profile_rep.TOTPOP_CY
1 HHPOP_CY.2021 Key Demographic Indicators (Esri... populationtotals.HHPOP_CY
2 FAMPOP_CY.2021 Key Demographic Indicators (Esr... populationtotals.FAMPOP_CY
3 MP14129a_B.2021 Health (Market Potential) (Esr... HealthPersonalCare.MP14129a_B
4 X11001_X.2021 Education (Consumer Spending) (E... education.X11001_X
5 X11001_A.2021 Education (Consumer Spending) (E... education.X11001_A
6 X11001_I.2021 Education (Consumer Spending) (E... Community_Profile_rep.X11001_I
7 MP09147a_B.2021 Electronics (Market Potential)... ElectronicsInternet.MP09147a_B
8 MP09147a_I.2021 Electronics (Market Potential)... ElectronicsInternet.MP09147a_I
9 MP09148a_B.2021 Electronics (Market Potential)... ElectronicsInternet.MP09148a_B
10 MP09148a_I.2021 Electronics (Market Potential)... ElectronicsInternet.MP09148a_I
11 MP09134a_B.2021 Electronics (Market Potential)... ElectronicsInternet.MP09134a_B
12 MP09134a_I.2021 Electronics (Market Potential)... ElectronicsInternet.MP09134a_I
13 ACSTOTPOP.2015-2019 Key Demographic Indicators... population.ACSTOTPOP
14 MOETOTPOP.2015-2019 Key Demographic Indicators... ACS_Housing_Summary_rep.MOETOTPOP
15 RELTOTPOP.2015-2019 Key Demographic Indicators... population.RELTOTPOP
16 INDAGRI_CY.2021 Labor Force (Esri) (US 2021 Re... industry.INDAGRI_CY
17 INDMIN_CY.2021 Labor Force (Esri) (US 2021 Ret... industry.INDMIN_CY
18 INDCONS_CY.2021 Labor Force (Esri) (US 2021 Re... industry.INDCONS_CY
19 HHWHTHHR10.2010 Households by Race_Hispanic Or... householdsbyraceofhouseholder.HHWHTHHR10
20 HHBLKHHR10.2010 Households by Race_Hispanic Or... householdsbyraceofhouseholder.HHBLKHHR10
21 HHAIHHR10.2010 Households by Race_Hispanic Ori... householdsbyraceofhouseholder.HHAIHHR10
22 HHASNHHR10.2010 Households by Race_Hispanic Or... householdsbyraceofhouseholder.HHASNHHR10
23 HHPIHHR10.2010 Households by Race_Hispanic Ori... householdsbyraceofhouseholder.HHPIHHR10
24 HHOTHHHR10.2010 Households by Race_Hispanic Or... householdsbyraceofhouseholder.HHOTHHHR10
25 TADULTBASE.2021 Tapestry Market Segmentation (... travelMPI.TADULTBASE
26 TADULT01.2021 Tapestry Market Segmentation (Ad... tapestryadultsNEW.TADULT01
27 TADULT02.2021 Tapestry Market Segmentation (Ad... tapestryadultsNEW.TADULT02
28 POPGRW10CY.2021 Key Demographic Indicators (Es... KeyUSFacts.POPGRW10CY
29 DIVINDX_CY.2021 Race and Hispanic Origin (Esri... Policy.DIVINDX_CY
30 MEDHINC_CY.2021 Income (Esri) (Esri USA 2021) Health.MEDHINC_CY
31 ACSMEDHINC.2015-2019 Income (ACS) (Esri USA 20... householdincome.ACSMEDHINC
32 MOEMEDHINC.2015-2019 Income (ACS) (Esri USA 20... householdincome.MOEMEDHINC
33 RELMEDHINC.2015-2019 Income (ACS) (Esri USA 20... householdincome.RELMEDHINC

Saving Results

Obviously, copying and pasting out of this notebook is a possibility, but Pandas also includes a large number of output options including back to Excel or one of my personal favorites, directly to the clipboard.

Input/Output - Pandas Documentation

Garbage Disposal Function

Taking advantage of a little iteration we can create a more universal function to try and find matches based on any of these input types. While we are at it, we can also add a bit of convenience, the option to copy the results directly to the clipboard.

[18]:
from pathlib import Path
from typing import Union, Iterable

from arcgis.geoenrichment import Country
from pandas.io.clipboards import to_clipboard


def get_variable_string(input_names: Union[Iterable[str], Path], country: Country, copy_to_clipboard: bool = False) -> str:
    """
    Based on an iterable input identifying variables, return the string needed to perform enrichment using the Enrich Layer
    geoprocessing tool.

    Args:
        input_names: Iterable (typically a list) of strings identifying the variable names for enrichment, or a path to a
            Feature Class if trying to find variable names from the feature inputs.
        country: Country to search in for enrichment variables.
        copy_to_clipboard: Whether or not to copy the result to the clipboard. Default is false.

    Return:
        String formatted for input into the Enrich Layer geoprocessing tool.
    """
    # if the input names is a path to a feature class, get the column names to use for lookup
    if isinstance(input_names, Path):
        input_names = [f.name for f in arcpy.ListFields(input_names)]

    # try to find valid enrich variable strings based on the input names
    enrich_vars = country._ba_cntry.get_enrich_variables_from_iterable(input_names)

    # if nothing found, try to split on a period, and see if this provides any matches
    if len(enrich_vars.index) == 0:
        input_names = [var_str.split('.')[0] for var_str in input_names]

        # try again - NOTE: not doing the split before trying, becuase another potential input format can include periods
        enrich_vars = country._ba_cntry.get_enrich_variables_from_iterable(input_names)

    # sort for consistency
    enrich_vars.sort_values('name', inplace=True)

    # combine the enrich variable strings into a single string
    enrich_str = ';'.join(enrich_vars.enrich_name.values)

    # now, if desiring making life easy by dropping right into the clipboard
    if copy_to_clipboard:
        to_clipboard(enrich_str, excel=False)

    return enrich_str

Now, with this new function we can input the variations from above, and consistently get the results we need. Because of the repeat trying to find variables, you will encounter the warning below if passing in the variables needing to be modified (split on the period).

[19]:
get_variable_string(xlsx_df.guided_workflow_names, country=usa)
d:\projects\geosaurus\src\arcgis\geoenrichment\_business_analyst\_main.py:1157: UserWarning: It appears none of the input enrich enrich_variables were found.
  warn(f"It appears none of the input enrich enrich_variables were found.")
[19]:
'householdincome.ACSMEDHINC;population.ACSTOTPOP;Policy.DIVINDX_CY;populationtotals.FAMPOP_CY;householdsbyraceofhouseholder.HHAIHHR10;householdsbyraceofhouseholder.HHASNHHR10;householdsbyraceofhouseholder.HHBLKHHR10;householdsbyraceofhouseholder.HHOTHHHR10;householdsbyraceofhouseholder.HHPIHHR10;populationtotals.HHPOP_CY;householdsbyraceofhouseholder.HHWHTHHR10;industry.INDAGRI_CY;industry.INDCONS_CY;industry.INDMIN_CY;Health.MEDHINC_CY;householdincome.MOEMEDHINC;ACS_Housing_Summary_rep.MOETOTPOP;ElectronicsInternet.MP09134a_B;ElectronicsInternet.MP09134a_I;ElectronicsInternet.MP09147a_B;ElectronicsInternet.MP09147a_I;ElectronicsInternet.MP09148a_B;ElectronicsInternet.MP09148a_I;HealthPersonalCare.MP14129a_B;KeyUSFacts.POPGRW10CY;householdincome.RELMEDHINC;population.RELTOTPOP;tapestryadultsNEW.TADULT01;tapestryadultsNEW.TADULT02;travelMPI.TADULTBASE;Age_by_Sex_by_Race_Profile_rep.TOTPOP_CY;education.X11001_A;Community_Profile_rep.X11001_I;education.X11001_X'
[20]:
get_variable_string(xlsx_df.fieldnames_arcmap, country=usa)
[20]:
'householdincome.ACSMEDHINC;population.ACSTOTPOP;Policy.DIVINDX_CY;populationtotals.FAMPOP_CY;householdsbyraceofhouseholder.HHAIHHR10;householdsbyraceofhouseholder.HHASNHHR10;householdsbyraceofhouseholder.HHBLKHHR10;householdsbyraceofhouseholder.HHOTHHHR10;householdsbyraceofhouseholder.HHPIHHR10;populationtotals.HHPOP_CY;householdsbyraceofhouseholder.HHWHTHHR10;industry.INDAGRI_CY;industry.INDCONS_CY;industry.INDMIN_CY;Health.MEDHINC_CY;householdincome.MOEMEDHINC;ACS_Housing_Summary_rep.MOETOTPOP;ElectronicsInternet.MP09134a_B;ElectronicsInternet.MP09134a_I;ElectronicsInternet.MP09147a_B;ElectronicsInternet.MP09147a_I;ElectronicsInternet.MP09148a_B;ElectronicsInternet.MP09148a_I;HealthPersonalCare.MP14129a_B;KeyUSFacts.POPGRW10CY;householdincome.RELMEDHINC;population.RELTOTPOP;tapestryadultsNEW.TADULT01;tapestryadultsNEW.TADULT02;travelMPI.TADULTBASE;Age_by_Sex_by_Race_Profile_rep.TOTPOP_CY;education.X11001_A;Community_Profile_rep.X11001_I;education.X11001_X'
[21]:
get_variable_string(xlsx_df.pro_enrich_names, country=usa)
[21]:
'householdincome.ACSMEDHINC;veterans.ACSTOTPOP;raceandhispanicorigin.DIVINDX_CY;populationtotals.FAMPOP_CY;householdsbyraceofhouseholder.HHAIHHR10;householdsbyraceofhouseholder.HHASNHHR10;householdsbyraceofhouseholder.HHBLKHHR10;householdsbyraceofhouseholder.HHOTHHHR10;householdsbyraceofhouseholder.HHPIHHR10;householdsbyraceofhouseholder.HHWHTHHR10;industry.INDAGRI_CY;industry.INDCONS_CY;industry.INDMIN_CY;householdincome.MEDHINC_CY;ElectronicsInternet.MP09134a_B;ElectronicsInternet.MP09147a_B;ElectronicsInternet.MP09148a_B;HealthPersonalCare.MP14129a_B;populationtotals.POPGRW10CY;tapestryadultsNEW.TADULT01;tapestryadultsNEW.TADULT02;tapestryadultsNEW.TADULTBASE;householdtotals.TOTHH_CY;populationtotals.TOTPOP_CY;education.X11001_X'