Filter records

Filter records to remove records invalid Pickup DateTime

In the interest of time, we will filter the records between October 2020 and December 2020 to reduce job processing time during the workshop.

  1. Click on the Transform icon, choose Filter.

  2. Specify the following information:

    • Node properties tab, Name – Filter - Yellow Trip Data
    • Transform tab, Filter condition – tpep_pickup_datetime matches ^2020-1

    Glue Jobs

  3. Remember to Save your work.

Review the automatically generated script

  1. Go to the Script details tab.

  2. Observe that the code of the script are automatically generated by Glue Studio.

    Glue Jobs