9.1 Python for data analysis and visualization

Python is a powerhouse for data journalism, offering tools to crunch numbers and create eye-catching visuals. It's like having a Swiss Army knife for data - you can slice, dice, and present information in ways that make complex stories accessible to everyone.

With libraries like Pandas, NumPy, Matplotlib, and Seaborn, Python transforms raw data into compelling narratives. These tools help journalists uncover hidden patterns, create stunning charts, and turn dry statistics into engaging stories that captivate readers and drive home important points.

Python Syntax and Data Types

Basic Python Syntax

Top images from around the web for Basic Python Syntax

• 

  Copy-paste into Python interactive interpreter and indentation - Stack Overflow View original

  Is this image relevant?

• 

  Copy-paste into Python interactive interpreter and indentation - Stack Overflow View original

  Is this image relevant?

1 of 1

Top images from around the web for Basic Python Syntax

• 

  Copy-paste into Python interactive interpreter and indentation - Stack Overflow View original

  Is this image relevant?

• 

  Copy-paste into Python interactive interpreter and indentation - Stack Overflow View original

  Is this image relevant?

1 of 1

• Python is a high-level, interpreted programming language that emphasizes code readability and simplicity
  • Uses indentation to define code blocks (rather than curly braces or keywords)
  • Has a relatively straightforward syntax compared to other programming languages (such as C++ or Java)
• Python uses dynamic typing, meaning that variables can hold values of different types, and the type of a variable can change during runtime
  • Type inference is used to determine the type of a variable based on the assigned value (no need to explicitly declare variable types)
• Python provides control flow statements to control the execution flow of the program based on conditions and iterations
  • if-else statements for conditional execution
  • for loops for iterating over sequences or ranges
  • while loops for repeating a block of code as long as a condition is true
• Functions in Python are defined using the

  def

  keyword, followed by the function name, parentheses containing optional parameters, and a colon
  • The function body is indented below the function definition
  • Functions can return values using the

    return

    statement

Data Types in Python

• Python supports several built-in data types, including:
  • Numeric types:

    int

    (integers),

    float

    (floating-point numbers),

    complex

    (complex numbers)
    • Support various arithmetic operations (addition, subtraction, multiplication, division, etc.)
  • Sequences:

    list

    (mutable, ordered collection),

    tuple

    (immutable, ordered collection),

    range

    (immutable sequence of numbers)
    • Can be accessed by their index (starting from 0)
  • Text type:

    str

    (represents a sequence of characters)
    • Supports string manipulation operations (concatenation, slicing, formatting, etc.)
  • Binary types:

    bytes

    (immutable sequence of bytes),

    bytearray

    (mutable sequence of bytes)
    • Used for handling raw binary data (such as images or network packets)
  • Mapping type:

    dict

    (unordered collection of key-value pairs)
    • Each key in a dictionary is unique
    • Allows efficient lookup of values based on their keys
  • Set types:

    set

    (mutable, unordered collection of unique elements),

    frozenset

    (immutable version of

    set

    )
    • Support mathematical set operations (union, intersection, difference, etc.)

Data Manipulation with Pandas and NumPy

Data Manipulation with Pandas

• Pandas is a powerful data manipulation library in Python that provides data structures and functions for working with structured data (such as tabular data in spreadsheets or SQL databases)
  • Primary data structures:

    [Series](https://www.fiveableKeyTerm:Series)

    (one-dimensional),

    [DataFrame](https://www.fiveableKeyTerm:dataframe)

    (two-dimensional)
    • Allow efficient storage and manipulation of labeled and typed data
  • Provides functions for reading and writing data from various file formats

    • read_csv()

      for CSV files,

      read_excel()

      for Excel files,

      read_json()

      for JSON files,

      read_sql()

      for SQL databases
  • Supports data cleaning and preprocessing functions

    • dropna()

      for handling missing values,

      fillna()

      for filling missing values,

      drop_duplicates()

      for removing duplicate rows,

      astype()

      for converting data types
  • Offers data transformation operations
    • Filtering, sorting, grouping, merging, and reshaping data
    • Functions like

      loc[]

      ,

      iloc[]

      ,

      sort_values()

      ,

      [groupby](https://www.fiveableKeyTerm:groupby)()

      ,

      merge()

      ,

      pivot()

      , and

      melt()

Numerical Computing with NumPy

• NumPy is a fundamental library for scientific computing in Python
  • Provides support for large, multi-dimensional arrays and matrices
  • Offers a collection of mathematical functions to operate on these arrays efficiently
• NumPy arrays are homogeneous (all elements have the same data type)
  • Allows for efficient memory usage and fast computations
• Provides functions for creating arrays

  • np.array()

    for creating arrays from lists or tuples

  • np.zeros()

    for creating arrays filled with zeros

  • np.ones()

    for creating arrays filled with ones

  • np.arange()

    for creating arrays with evenly spaced values

  • np.linspace()

    for creating arrays with a specified number of evenly spaced elements
• Supports array manipulation functions

  • reshape()

    for changing the shape of an array

  • transpose()

    for transposing an array

  • flatten()

    for converting a multi-dimensional array into a one-dimensional array
• Offers mathematical operations on arrays
  • Element-wise arithmetic operations using operators like

    +

    ,

    -

    ,

    *

    ,

    /

  • Functions like

    np.sum()

    ,

    np.[mean](https://www.fiveableKeyTerm:Mean)()

    ,

    np.std()

    ,

    np.min()

    ,

    np.max()

    ,

    np.median()

    for computing summary statistics
• Supports broadcasting (allows arrays with different shapes to be used in arithmetic operations without explicit loops)
  • Makes the code more concise and efficient

Data Visualization with Matplotlib and Seaborn

Data Visualization with Matplotlib

• Matplotlib is a fundamental plotting library in Python
  • Provides a MATLAB-like interface for creating a wide range of static, animated, and interactive visualizations
• Allows creating various types of plots
  • Line plots (

    plt.plot()

    ), scatter plots (

    plt.scatter()

    ), bar plots (

    plt.bar()

    ), histograms (

    plt.hist()

    ), box plots (

    plt.boxplot()

    ), heatmaps (

    plt.imshow()

    )
• Supports plot customization
  • Setting properties like titles (

    plt.title()

    ), labels (

    plt.xlabel()

    ,

    plt.ylabel()

    ), legends (

    plt.legend()

    ), colors, markers, linestyles, and font sizes
  • Changing plot styles using

    plt.style.use()

• Enables creating subplots to display multiple plots in a single figure
  • Functions like

    plt.subplot()

    and

    plt.subplots()

    for efficient comparison and analysis of different datasets or plot types
• Allows saving figures to various file formats
  • PNG, JPEG, PDF, and SVG using

    plt.savefig()

  • Enables inclusion of high-quality visualizations in reports, presentations, and web pages

Statistical Data Visualization with Seaborn

• Seaborn is a statistical data visualization library built on top of Matplotlib
  • Provides a high-level interface for creating informative and attractive statistical graphics
• Offers functions for visualizing univariate, bivariate, and multivariate relationships in data

  • sns.histplot()

    for histograms,

    sns.kdeplot()

    for kernel density plots,

    sns.scatterplot()

    for scatter plots,

    sns.lineplot()

    for line plots,

    sns.barplot()

    for bar plots,

    sns.heatmap()

    for heatmaps
• Supports visualizing categorical data
  • Functions like

    sns.catplot()

    ,

    sns.boxplot()

    ,

    sns.violinplot()

    , and

    sns.swarmplot()

  • Automatically handles grouping and aggregation of data based on categorical variables
• Allows easy customization of plot aesthetics
  • Built-in themes and color palettes using functions like

    sns.set_theme()

    and

    sns.set_palette()

  • Ensures consistent and visually appealing plots across a project
• Integrates well with Pandas data structures
  • Allows direct plotting of

    DataFrame

    and

    Series

    objects without explicit data extraction or transformation

Python for Data Journalism

Data Journalism Workflow

• Data journalism involves using data analysis and visualization techniques to uncover insights, patterns, and stories in datasets and communicate them effectively to the audience
• Typical data journalism workflow includes:
  1. Data acquisition: Obtaining relevant datasets from sources like government databases, public APIs, web scraping, or freedom of information requests
  2. Data cleaning and preprocessing: Handling missing values, removing duplicates, converting data types, and transforming data using libraries like Pandas
  3. Exploratory data analysis (EDA): Examining dataset's structure, summary statistics, and relationships between variables using Pandas, NumPy, Matplotlib, and Seaborn
  4. Data analysis: Applying statistical techniques, machine learning algorithms, or custom data manipulations to extract meaningful insights and patterns
  5. Data visualization: Creating informative and engaging visualizations using Matplotlib, Seaborn, or interactive libraries like Plotly or Bokeh
  6. Storytelling: Combining data insights with contextual information, expert opinions, and human stories to create a compelling narrative
• Real-world data journalism projects often involve working with large, complex, and messy datasets
  • Requires skills in data wrangling, cleaning, and integration using Python libraries like Pandas, NumPy, and regular expressions

Collaboration and Publishing in Data Journalism

• Collaborative data journalism projects may involve using version control systems like Git and platforms like GitHub
  • Facilitates teamwork, tracks changes, and ensures reproducibility of the analysis and visualizations
• Publishing data journalism projects may require:
  • Exporting visualizations to suitable formats
  • Creating interactive web-based visualizations using libraries like Plotly or Bokeh
  • Integrating the analysis and visualizations into content management systems or web frameworks