Skip to content
Proposal for a new site by Genesis Innovation, built from his public LinkedIn profile. Not his official site.

Wiki Python for data

Dates, times and time zones

A timestamp without a time zone is ambiguous. Store in UTC, convert when displaying, and mind the difference between a date and an instant.

A datetime can be "naive" (no time zone) or "aware" (with a time zone). Comparing the two kinds raises an error, and that is a good thing: mixing them is a bug.

PythonOutput checked by running Python 3
from datetime import datetime, timedelta, timezone

brasilia = timezone(timedelta(hours=-3))
event = datetime(2024, 6, 1, 23, 30, tzinfo=brasilia)
print(event.isoformat())
print(event.astimezone(timezone.utc).isoformat())
print(event.date(), event.astimezone(timezone.utc).date())

Output (checked)

2024-06-01T23:30:00-03:00
2024-06-02T02:30:00+00:00
2024-06-01 2024-06-02

The same instant falls on different days depending on the time zone. A "sales of the day" report has to say which time zone the day is in.

Epoch

Systems exchange instants as seconds (or milliseconds) since 1970-01-01 UTC.

PythonOutput checked by running Python 3
from datetime import datetime, timezone

seconds = 1717286400
print(datetime.fromtimestamp(seconds, tz=timezone.utc).isoformat())
print(int(datetime(2024, 6, 2, tzinfo=timezone.utc).timestamp() * 1000))

Output (checked)

2024-06-02T00:00:00+00:00
1717286400000

Parsing and formatting

PythonOutput checked by running Python 3
from datetime import datetime

d = datetime.strptime("05/06/2024 14:07", "%d/%m/%Y %H:%M")
print(d.strftime("%Y-%m-%d %H:%M:%S"))
print(d.weekday(), d.strftime("%j"))

Output (checked)

2024-06-05 14:07:00
2 157

weekday() counts Monday as 0: 2 is Wednesday. %j is the day of the year.

  • For time zones with daylight saving time, use zoneinfo.ZoneInfo("America/Sao_Paulo") (PEP 615), which reads the system's time zone database. Brazil has had no daylight saving time since 2019, but old data does.
  • In Spark, spark.sql.session.timeZone decides how timestamp is displayed and converted: pin that setting in the job instead of inheriting the machine's.

Sources

Shared knowledge from Genesis Innovation. Not a claim about the person behind this site.