4. Basic example of pySocialWatcher¶
At the time of writing this documentation, the Facebook Marketing API version was 9.0 and that is precisely the API version used in the token that we have generated before. Assure that the code below and the version used in your token are adjusted accordingly.
The start point of pySocialWatcher is the method watcherAPI. watcherAPI has a few optional parameters. In the following example, we use the parameter outputname, which allows us to define the filename where the resulting collection will be saved.
from pysocialwatcher import watcherAPI
watcher = watcherAPI(api_version="9.0", outputname="output_quick_start_collection.csv")
We need to create a credential file where we store, respectively, the userId and access token used in the data collection as follows:
> cat credentials.csv
EAAM7F8Gp66MBAHlP0QB5sYZCzpdKLaDGowh6XM30EXMlK7DaKtrH0LbGsyUQZB4xMFz84Xjw6lSeT3no3yy8yWOPN46UifRxLctZCyaZBR6tapNwARGQTHhdU1kMBTo30nf92ANRRhtea4FpD2ZCDwzOvlTqxhpolowiVbashUXXXXX,320000000000291
Multiple pairs of <userId, token> can be store in the same file, allowing pySocialWatcher to issue multiple API requests at once.
To load the credential file (e.g., credentials.csv), we need to use the following command:
watcher.load_credentials_file("credentials.csv")
Now, we are ready to run our very first collection.
Multiple examples of collections can be found online and we will explore how to build our own customizable collection on another notebook.
This is done with a json file as below:
{
"name": "Total Population",
"geo_locations": [
{ "name": "countries", "values": ["BR"] },
{ "name": "countries", "values": ["IT"] }
],
"genders": [0,1,2],
"ages_ranges": [
{"min":18}
]
}
In this example, saved on disk as quick_example.json, we retrieve the estimated number of Facebook users at least 18 years old desegregated by gender for two locations, Brazil and Italy.
In total, we are going to issue 6 API calls (2 locations (Brazil and Italy) * 3 genders (male, female, both) * 1 age bucket (18+)).
After the collection is finished, the result is both saved as a dataframe on disk (output_quick_start_collection, as we specified before) as well as returned as a pandas dataframe object.
This is a very quick collection and should not take longer than one minute, but long collections might take weeks or even months.
df = watcher.run_data_collection("quick_example.json", remove_tmp_files=True)
2021-01-30 19:35:41 donna root[1824288] INFO Building Collection Dataframe
2021-01-30 19:35:41 donna root[1824288] WARNING Field not expecified: interests
2021-01-30 19:35:41 donna root[1824288] WARNING Field not expecified: behavior
2021-01-30 19:35:41 donna root[1824288] WARNING Field not expecified: scholarities
2021-01-30 19:35:41 donna root[1824288] WARNING Field not expecified: languages
2021-01-30 19:35:41 donna root[1824288] WARNING Field not expecified: family_statuses
2021-01-30 19:35:41 donna root[1824288] WARNING Field not expecified: relationship_statuses
2021-01-30 19:35:41 donna root[1824288] WARNING Field not expecified: household_composition
2021-01-30 19:35:41 donna root[1824288] INFO Total API Requests:6
2021-01-30 19:35:41 donna root[1824288] INFO Completed: 0.00
2021-01-30 19:35:41 donna root[1824288] WARNING No field: languages
2021-01-30 19:35:41 donna root[1824288] INFO Completed: 16.67
2021-01-30 19:35:41 donna root[1824288] WARNING No field: languages
2021-01-30 19:35:41 donna root[1824288] INFO Completed: 33.33
2021-01-30 19:35:41 donna root[1824288] WARNING No field: languages
2021-01-30 19:35:41 donna root[1824288] INFO Completed: 50.00
2021-01-30 19:35:41 donna root[1824288] WARNING No field: languages
2021-01-30 19:35:41 donna root[1824288] INFO Completed: 66.67
2021-01-30 19:35:41 donna root[1824288] WARNING No field: languages
2021-01-30 19:35:41 donna root[1824288] INFO Completed: 83.33
2021-01-30 19:35:41 donna root[1824288] WARNING No field: languages
2021-01-30 19:35:41 donna root[1824288] INFO Saving Skeleton file: dataframe_skeleton_1611497659.csv.gz
2021-01-30 19:35:41 donna root[1824288] INFO Collecting... Completed: 0.00% , 0/6
2021-01-30 19:35:41 donna root[2170838] WARNING Sending in request: {'optimization_goal': 'AD_RECALL_LIFT', 'targeting_spec': '{"geo_locations": {"countries": ["BR"], "location_types": ["home"]}, "age_min": 18, "age_max": null, "genders": [0], "flexible_spec": [], "publisher_platforms": ["facebook"]}', 'access_token': 'EAAM7F8Gp66MBADlruTZBfHkPXvXviZB1iPmhDK9beEjVUk1iGQKIR161ecHMZActHaC8QWxRewZBa2vqidk6qr2ZAirVcD9zLU6vUQrep6R0rZAs3yJygcRLl9n1ZAkLHfiEYZCbBbISfMGV2ZCSfXIJUggmJ0yHygo11Rv77tZBem8wZDZD'}
2021-01-30 19:35:42 donna root[1824288] INFO Collecting... Completed: 16.67% , 1/6
2021-01-30 19:35:50 donna root[2170882] WARNING Sending in request: {'optimization_goal': 'AD_RECALL_LIFT', 'targeting_spec': '{"geo_locations": {"countries": ["IT"], "location_types": ["home"]}, "age_min": 18, "age_max": null, "genders": [0], "flexible_spec": [], "publisher_platforms": ["facebook"]}', 'access_token': 'EAAM7F8Gp66MBADlruTZBfHkPXvXviZB1iPmhDK9beEjVUk1iGQKIR161ecHMZActHaC8QWxRewZBa2vqidk6qr2ZAirVcD9zLU6vUQrep6R0rZAs3yJygcRLl9n1ZAkLHfiEYZCbBbISfMGV2ZCSfXIJUggmJ0yHygo11Rv77tZBem8wZDZD'}
2021-01-30 19:35:51 donna root[1824288] INFO Collecting... Completed: 33.33% , 2/6
2021-01-30 19:35:59 donna root[2170912] WARNING Sending in request: {'optimization_goal': 'AD_RECALL_LIFT', 'targeting_spec': '{"geo_locations": {"countries": ["BR"], "location_types": ["home"]}, "age_min": 18, "age_max": null, "genders": [1], "flexible_spec": [], "publisher_platforms": ["facebook"]}', 'access_token': 'EAAM7F8Gp66MBADlruTZBfHkPXvXviZB1iPmhDK9beEjVUk1iGQKIR161ecHMZActHaC8QWxRewZBa2vqidk6qr2ZAirVcD9zLU6vUQrep6R0rZAs3yJygcRLl9n1ZAkLHfiEYZCbBbISfMGV2ZCSfXIJUggmJ0yHygo11Rv77tZBem8wZDZD'}
2021-01-30 19:36:00 donna root[1824288] INFO Collecting... Completed: 50.00% , 3/6
2021-01-30 19:36:08 donna root[2170957] WARNING Sending in request: {'optimization_goal': 'AD_RECALL_LIFT', 'targeting_spec': '{"geo_locations": {"countries": ["IT"], "location_types": ["home"]}, "age_min": 18, "age_max": null, "genders": [1], "flexible_spec": [], "publisher_platforms": ["facebook"]}', 'access_token': 'EAAM7F8Gp66MBADlruTZBfHkPXvXviZB1iPmhDK9beEjVUk1iGQKIR161ecHMZActHaC8QWxRewZBa2vqidk6qr2ZAirVcD9zLU6vUQrep6R0rZAs3yJygcRLl9n1ZAkLHfiEYZCbBbISfMGV2ZCSfXIJUggmJ0yHygo11Rv77tZBem8wZDZD'}
2021-01-30 19:36:09 donna root[1824288] INFO Collecting... Completed: 66.67% , 4/6
2021-01-30 19:36:16 donna root[2170995] WARNING Sending in request: {'optimization_goal': 'AD_RECALL_LIFT', 'targeting_spec': '{"geo_locations": {"countries": ["BR"], "location_types": ["home"]}, "age_min": 18, "age_max": null, "genders": [2], "flexible_spec": [], "publisher_platforms": ["facebook"]}', 'access_token': 'EAAM7F8Gp66MBADlruTZBfHkPXvXviZB1iPmhDK9beEjVUk1iGQKIR161ecHMZActHaC8QWxRewZBa2vqidk6qr2ZAirVcD9zLU6vUQrep6R0rZAs3yJygcRLl9n1ZAkLHfiEYZCbBbISfMGV2ZCSfXIJUggmJ0yHygo11Rv77tZBem8wZDZD'}
2021-01-30 19:36:16 donna root[1824288] INFO Collecting... Completed: 83.33% , 5/6
2021-01-30 19:36:25 donna root[2171087] WARNING Sending in request: {'optimization_goal': 'AD_RECALL_LIFT', 'targeting_spec': '{"geo_locations": {"countries": ["IT"], "location_types": ["home"]}, "age_min": 18, "age_max": null, "genders": [2], "flexible_spec": [], "publisher_platforms": ["facebook"]}', 'access_token': 'EAAM7F8Gp66MBADlruTZBfHkPXvXviZB1iPmhDK9beEjVUk1iGQKIR161ecHMZActHaC8QWxRewZBa2vqidk6qr2ZAirVcD9zLU6vUQrep6R0rZAs3yJygcRLl9n1ZAkLHfiEYZCbBbISfMGV2ZCSfXIJUggmJ0yHygo11Rv77tZBem8wZDZD'}
2021-01-30 19:36:25 donna root[1824288] INFO Data Collection Complete
2021-01-30 19:36:25 donna root[1824288] INFO Saving temporary file: dataframe_collecting_1611497659.csv.gz
2021-01-30 19:36:25 donna root[1824288] INFO Computing Audience and DAU column
2021-01-30 19:36:25 donna root[1824288] INFO Saving after collecting file: output_quick_start_collection.csv
df.head(6)
| name | interests | ages_ranges | genders | behavior | scholarities | languages | family_statuses | relationship_statuses | geo_locations | household_composition | all_fields | targeting | response | dau_audience | mau_audience | timestamp | publisher_platforms | mock_response | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | Total Population | NaN | {'min': 18} | 0 | NaN | NaN | NaN | NaN | NaN | {'name': 'countries', 'values': ['BR']} | NaN | ((ages_ranges, {'min': 18}), (genders, 0), (ge... | {'geo_locations': {'countries': ['BR'], 'locat... | b'{"data":[{"daily_outcomes_curve":[{"spend":0... | 78883199 | 120000000 | 1611497659 | ["facebook"] | False |
| 1 | Total Population | NaN | {'min': 18} | 0 | NaN | NaN | NaN | NaN | NaN | {'name': 'countries', 'values': ['IT']} | NaN | ((ages_ranges, {'min': 18}), (genders, 0), (ge... | {'geo_locations': {'countries': ['IT'], 'locat... | b'{"data":[{"daily_outcomes_curve":[{"spend":0... | 24302759 | 31000000 | 1611497659 | ["facebook"] | False |
| 2 | Total Population | NaN | {'min': 18} | 1 | NaN | NaN | NaN | NaN | NaN | {'name': 'countries', 'values': ['BR']} | NaN | ((ages_ranges, {'min': 18}), (genders, 1), (ge... | {'geo_locations': {'countries': ['BR'], 'locat... | b'{"data":[{"daily_outcomes_curve":[{"spend":0... | 35017919 | 56000000 | 1611497659 | ["facebook"] | False |
| 3 | Total Population | NaN | {'min': 18} | 1 | NaN | NaN | NaN | NaN | NaN | {'name': 'countries', 'values': ['IT']} | NaN | ((ages_ranges, {'min': 18}), (genders, 1), (ge... | {'geo_locations': {'countries': ['IT'], 'locat... | b'{"data":[{"daily_outcomes_curve":[{"spend":0... | 11433599 | 15000000 | 1611497659 | ["facebook"] | False |
| 4 | Total Population | NaN | {'min': 18} | 2 | NaN | NaN | NaN | NaN | NaN | {'name': 'countries', 'values': ['BR']} | NaN | ((ages_ranges, {'min': 18}), (genders, 2), (ge... | {'geo_locations': {'countries': ['BR'], 'locat... | b'{"data":[{"daily_outcomes_curve":[{"spend":0... | 44095999 | 65000000 | 1611497659 | ["facebook"] | False |
| 5 | Total Population | NaN | {'min': 18} | 2 | NaN | NaN | NaN | NaN | NaN | {'name': 'countries', 'values': ['IT']} | NaN | ((ages_ranges, {'min': 18}), (genders, 2), (ge... | {'geo_locations': {'countries': ['IT'], 'locat... | b'{"data":[{"daily_outcomes_curve":[{"spend":0... | 12151799 | 15000000 | 1611497659 | ["facebook"] | False |
The most relevant columns are:
“mau_audience” (MAU = Monthly Active Users): The expected number of active FaceBook users that will supposedly see our advertising in the following month.
“dau_audience” (DAU = Daily Active Users): The expected number of active FaceBook users that will supposedly see our advertising in the following day. This number is considerably smaller than the MAU and probably too unstable to work with.
“targeting”: This column has all the details of your collection. Other columns in the dataframe derive from it. For example, note the columns
gendersandgeo_locationsin the collection above.
df[["targeting", "dau_audience", "mau_audience"]]
| targeting | dau_audience | mau_audience | |
|---|---|---|---|
| 0 | {'geo_locations': {'countries': ['BR'], 'locat... | 78883199 | 120000000 |
| 1 | {'geo_locations': {'countries': ['IT'], 'locat... | 24302759 | 31000000 |
| 2 | {'geo_locations': {'countries': ['BR'], 'locat... | 35017919 | 56000000 |
| 3 | {'geo_locations': {'countries': ['IT'], 'locat... | 11433599 | 15000000 |
| 4 | {'geo_locations': {'countries': ['BR'], 'locat... | 44095999 | 65000000 |
| 5 | {'geo_locations': {'countries': ['IT'], 'locat... | 12151799 | 15000000 |
What exactly we are sending to Facebook Marketing API is a requests like the following json from the targeting column:
df["targeting"].loc[0]
{'geo_locations': {'countries': ['BR'], 'location_types': ['home']},
'age_min': 18,
'age_max': None,
'genders': [0],
'flexible_spec': [],
'publisher_platforms': ['facebook']}
Note that the output is not yet ready to be used for excel spreadsheets or to generate visualizations. For that, we will take advantage of new pySocialWatcher features to postprocess the collection. But, before diveing into it, we will learning on the next section how to easily create JSON files for more interesting data collections.