Thread

Power Query Date Table Generation and Refactoring

This thread delves into generating custom date tables, refactoring M code, and analyzing code in DOT notation in Power Query M language.

Empty image or helper icon

Power Query Date Table Generation and Refactoring

Description

This thread delves into generating custom date tables, refactoring M code, and analyzing code in DOT notation in Power Query M language.

Custom Date Table Generation for Fiscal Years in Power Query M

Tool: Code Explainer

Created: 03/22/2024

Prompt

let
    fxFY = ( years as list ) as table => [
        t = List.Transform( years, each [
            n = _ - List.First(years),
            FYs = List.Repeat( {_}, 52),
            FYID = List.Repeat( {n+1}, 52),
            Weeks = {1..52},
            FWID = List.Transform( Weeks, (x)=> (52 * n) +x ),
            FYPeriod = List.Transform({0..51}, (x)=> Number.IntegerDivide(x, 4) +1),
            FPID = List.Transform(FYPeriod, (x)=> (13 * n) +x ),
            StartDate = List.Transform( {0..51}, (x)=> Date.StartOfWeek( Date.AddDays( #date(_, 4, 1), x *7), Day.Saturday )),
            EndDate = List.Transform( {0..51}, (x)=> Date.EndOfWeek( Date.AddDays( #date(_, 4, 1), x *7), Day.Saturday )),
            Dates = List.Transform( StartDate, (x)=> List.Dates(x, 7, Duration.From(1))),
            t = Table.FromColumns(
                {
                    FYs, FYID, Dates, Weeks, FWID, StartDate, EndDate, FYPeriod, FPID
                },  type table [
                    FY=Int64.Type, FY ID=Int64.Type, Date={date}, Week=Int64.Type, FW ID=Int64.Type, StartDate=date, EndDate=date, FY Period=Int64.Type, FP ID=Int64.Type
                ]
            )
        ][t] ),
        r = Table.ExpandListColumn( Table.Combine(t), "Date")
    ][r],
    // Invoke custom function on a list of year numbers
    Source = fxFY( {2020..2025} ), 
    // Use UI to extend this custom date table with CY attributes
    InsertYear = Table.AddColumn(Source, "CY", each Date.Year([StartDate]), Int64.Type)
in
    InsertYear

The provided code snippet is written in Power Query M, a functional language used in Power BI and Excel for data manipulation and transformations. The code generates a custom date table for fiscal years.

Code Analysis

The code uses a custom function, fxFY, passed with a list of years ({2020..2025}), and finally adds a new column to the produced table called CY that represents the year of the StartDate.

Code Structure
  1. Function Definition: The fxFY function takes a list of years as input and returns a table. It uses several core functions (List.Transform, List.First, List.Repeat, Table.FromColumns, Table.ExpandListColumn, Table.Combine) and also resolves list range elements ({1..52}, {0..51}).

  2. Function Invocation: The function fxFY is invoked with the list of years {2020..2025}.

  3. Column Addition: Table.AddColumn function is used to add a new column named CY, calculated from the StartDate field.

Key Functions
  • List.Transform: Applies a function to each item of the list and returns a list.
  • List.First : Returns the first item of the list.
  • List.Repeat: Repeats a value or a list of values.
  • Table.FromColumns: Creates a table from a list of columns.
  • Table.ExpandListColumn, Table.Combine: Manipulates and merges tables.
  • #date: Creates a date.

The function fxFY creates a list (t) where each year is represented by a table that includes fiscal year columns (like FYs, FYID, Weeks, FWID, FYPeriod, FPID, StartDate, EndDate, Dates). Then, it combines all these tables into one table and expands the column Date as it contains lists of dates.

Finally, the function Table.AddColumn is used to add a new column "CY" to the Source table by extracting the year part of the StartDate column. It uses the Date.Year function to extract the year. The output table will have data for years from 2020 to 2025.

Power Query Formula Language (M) Code Refactoring

Tool: Code Best Practices Checker

Created: 03/22/2024

Prompt

let
    fxFY = ( years as list ) as table => [
        t = List.Transform( years, each [
            n = _ - List.First(years),
            FYs = List.Repeat( {_}, 52),
            FYID = List.Repeat( {n+1}, 52),
            Weeks = {1..52},
            FWID = List.Transform( Weeks, (x)=> (52 * n) +x ),
            FYPeriod = List.Transform({0..51}, (x)=> Number.IntegerDivide(x, 4) +1),
            FPID = List.Transform(FYPeriod, (x)=> (13 * n) +x ),
            StartDate = List.Transform( {0..51}, (x)=> Date.StartOfWeek( Date.AddDays( #date(_, 4, 1), x *7), Day.Saturday )),
            EndDate = List.Transform( {0..51}, (x)=> Date.EndOfWeek( Date.AddDays( #date(_, 4, 1), x *7), Day.Saturday )),
            Dates = List.Transform( StartDate, (x)=> List.Dates(x, 7, Duration.From(1))),
            t = Table.FromColumns(
                {
                    FYs, FYID, Dates, Weeks, FWID, StartDate, EndDate, FYPeriod, FPID
                },  type table [
                    FY=Int64.Type, FY ID=Int64.Type, Date={date}, Week=Int64.Type, FW ID=Int64.Type, StartDate=date, EndDate=date, FY Period=Int64.Type, FP ID=Int64.Type
                ]
            )
        ][t] ),
        r = Table.ExpandListColumn( Table.Combine(t), "Date")
    ][r],
    // Invoke custom function on a list of year numbers
    Source = fxFY( {2020..2025} ), 
    // Use UI to extend this custom date table with CY attributes
    InsertYear = Table.AddColumn(Source, "CY", each Date.Year([StartDate]), Int64.Type)
in
    InsertYear
  1. Language/Technology Identification: The provided code snippet is written in Power Query Formula Language (M), a functional language used within Power BI, Microsoft's data visualization tool.

  2. Code Refactoring:

The original code seems to be clear, well-structured, and performs its intended function successfully. However, minor adjustments could enhance its readability and maintainability.

Here is the refactored code:

let
    createFYTable = ( years as list ) as table => [
        tbl = List.Transform( years, each [
                yearDiff = _ - List.First(years),
                FYs = List.Repeat( {_}, 52),
                FYID = List.Repeat( {yearDiff+1}, 52),
                Weeks = {1..52},
                FWID = List.Transform( Weeks, (week)=> (52 * yearDiff) + week ),
                FYPeriod = List.Transform({0..51}, (x)=> Number.IntegerDivide(x, 4) +1),
                FPID = List.Transform(FYPeriod, (period)=> (13 * yearDiff) + period ),
                StartDate = List.Transform( {0..51}, (x)=> Date.StartOfWeek( Date.AddDays( #date(_, 4, 1), x *7), Day.Saturday )),
                EndDate = List.Transform( {0..51}, (x)=> Date.EndOfWeek( Date.AddDays( #date(_, 4, 1), x *7), Day.Saturday )),
                Dates = List.Transform( StartDate, (start)=> List.Dates(start, 7, Duration.From(1))),
                tableColumns = Table.FromColumns(
                    {
                        FYs, FYID, Dates, Weeks, FWID, StartDate, EndDate, FYPeriod, FPID
                    },  
                    type table [
                        FY=Int64.Type, FY ID=Int64.Type, Date={date}, Week=Int64.Type, FW ID=Int64.Type, StartDate=date, EndDate=date, FY Period=Int64.Type, FP ID=Int64.Type
                    ]
                )
            ][tableColumns] ),
        result = Table.ExpandListColumn( Table.Combine(tbl), "Date")
    ][result],
    Source = createFYTable( {2020..2025} ), 
    AddYear = Table.AddColumn(Source, "CY", each Date.Year([StartDate]), Int64.Type)
in
    AddYear
  1. Documentation of Changes:

Few variables are renamed to have meaningful names such as fxFY is changed to createFYTable, t is to tbl, n to yearDiff and so on. This change will help in better comprehension of the code, enhancing its readability.

  1. Presentation in Code Blocks:

The original and refactored code are displayed in separate, clearly formatted code blocks. The changes are minimal but vital to improving the readability of the code and its ability to be maintained in the future.

Power Query M Custom Date Table Generator

Tool: Logic Visualizer

Created: 03/22/2024

Prompt

let
    fxFY = ( years as list ) as table => [
        t = List.Transform( years, each [
            n = _ - List.First(years),
            FYs = List.Repeat( {_}, 52),
            FYID = List.Repeat( {n+1}, 52),
            Weeks = {1..52},
            FWID = List.Transform( Weeks, (x)=> (52 * n) +x ),
            FYPeriod = List.Transform({0..51}, (x)=> Number.IntegerDivide(x, 4) +1),
            FPID = List.Transform(FYPeriod, (x)=> (13 * n) +x ),
            StartDate = List.Transform( {0..51}, (x)=> Date.StartOfWeek( Date.AddDays( #date(_, 4, 1), x *7), Day.Saturday )),
            EndDate = List.Transform( {0..51}, (x)=> Date.EndOfWeek( Date.AddDays( #date(_, 4, 1), x *7), Day.Saturday )),
            Dates = List.Transform( StartDate, (x)=> List.Dates(x, 7, Duration.From(1))),
            t = Table.FromColumns(
                {
                    FYs, FYID, Dates, Weeks, FWID, StartDate, EndDate, FYPeriod, FPID
                },  type table [
                    FY=Int64.Type, FY ID=Int64.Type, Date={date}, Week=Int64.Type, FW ID=Int64.Type, StartDate=date, EndDate=date, FY Period=Int64.Type, FP ID=Int64.Type
                ]
            )
        ][t] ),
        r = Table.ExpandListColumn( Table.Combine(t), "Date")
    ][r],
    // Invoke custom function on a list of year numbers
    Source = fxFY( {2020..2025} ), 
    // Use UI to extend this custom date table with CY attributes
    InsertYear = Table.AddColumn(Source, "CY", each Date.Year([StartDate]), Int64.Type)
in
    InsertYear

The illustrated visualization presents the logic and structure of the provided code snippet. The code is written in Power Query M language and is used for creating a custom date table for fiscal year calculations.

Visual Diagram

Start
|
|------> Invoke custom function (fxFY) 
|         |
|         |------> years (input)
|         |
|         |------> List.Transform (generates a list 't' of custom tables for each year)
|         |         |
|         |         |------------> Subtraction operation
|         |         |
|         |         |------------> List.Repeat (generates List of FYs and FYIDs)
|         |         |
|         |         |------------> List.Transform (generates FWIDs, FPIDs, StartDate, EndDate)
|         |         |
|         |         |------------> List of Dates
|         |         |
|         |         |------------> Create a Table From Columns
|         |
|         |------> Combined all tables in 't' into a single table 
|         |
|         |------> Expand the table along dates
|
|------> Add 'CY' column 
|
End

Annotations

  • Invoke Custom Function (fxFY): Process starts with invoking a personalized function that takes a list of years as input.

  • List.Transform & Subtraction operation: For every year, the program calculates the difference between the given year and the first year in the list.

  • List.Repeat: Using the calculated difference, a list of repeated year values (FYs) and FYID is created comprising of 52 repetitions.

  • List.Transform: Using the number of repetitions and calculated difference, lists of FWID, FPID, StartDate, EndDate are created.

  • List of Dates: A List of dates is created between start and end dates.

  • Create Table From Columns: All the generated lists are organized into columns to form a table.

  • Combined all tables in 't' into a single table: All the tables generated for each year are combined into a single table.

  • Expand the table along dates: The combined table is further expanded across the Date column.

  • Add 'CY' column: A column for CY is added to the expanded table, where each value is the year part of the StartDate in the corresponding row.

This diagram and annotations provide a simplified, step-by-step flow guiding the logic of the presented code from start to finish.

Code Analysis in DOT Notation

Tool: Code Visualizer

Created: 03/22/2024

Prompt

let
    fxFY = ( years as list ) as table => [
        t = List.Transform( years, each [
            n = _ - List.First(years),
            FYs = List.Repeat( {_}, 52),
            FYID = List.Repeat( {n+1}, 52),
            Weeks = {1..52},
            FWID = List.Transform( Weeks, (x)=> (52 * n) +x ),
            FYPeriod = List.Transform({0..51}, (x)=> Number.IntegerDivide(x, 4) +1),
            FPID = List.Transform(FYPeriod, (x)=> (13 * n) +x ),
            StartDate = List.Transform( {0..51}, (x)=> Date.StartOfWeek( Date.AddDays( #date(_, 4, 1), x *7), Day.Saturday )),
            EndDate = List.Transform( {0..51}, (x)=> Date.EndOfWeek( Date.AddDays( #date(_, 4, 1), x *7), Day.Saturday )),
            Dates = List.Transform( StartDate, (x)=> List.Dates(x, 7, Duration.From(1))),
            t = Table.FromColumns(
                {
                    FYs, FYID, Dates, Weeks, FWID, StartDate, EndDate, FYPeriod, FPID
                },  type table [
                    FY=Int64.Type, FY ID=Int64.Type, Date={date}, Week=Int64.Type, FW ID=Int64.Type, StartDate=date, EndDate=date, FY Period=Int64.Type, FP ID=Int64.Type
                ]
            )
        ][t] ),
        r = Table.ExpandListColumn( Table.Combine(t), "Date")
    ][r],
    // Invoke custom function on a list of year numbers
    Source = fxFY( {2020..2025} ), 
    // Use UI to extend this custom date table with CY attributes
    InsertYear = Table.AddColumn(Source, "CY", each Date.Year([StartDate]), Int64.Type)
in
    InsertYear

The given piece of code is in Power Query (M) Language used in Power BI and Excel. It generates a custom date table with a given range of years (2020 to 2025 in this case).

To visualize this code in DOT notation, we can organize it into nodes and edges to represent variables and their relationships.

    digraph G {
        // Node definitions
        fxFY [label="Function 'fxFY'"];
        Source [label="Variable 'Source'"];
        InsertYear [label="Variable 'InsertYear'"];
        t [label="Variable 't'"];
        n [label="Variable 'n'"];
        FYs [label="Variable 'FYs'"];
        FYID [label="Variable 'FYID'"];
        Weeks [label="Variable 'Weeks'"];
        FWID [label="Variable 'FWID'"];
        FYPeriod [label="Variable 'FYPeriod'"];
        FPID [label="Variable 'FPID'"];
        StartDate [label="Variable 'StartDate'"];
        EndDate [label="Variable 'EndDate'"];
        Dates [label="Variable 'Dates'"];
        
        // Relationship definitions
        fxFY -> t [label="Defines"];
        t -> n [label="Uses"];
        t -> FYs [label="Uses"];
        t -> FYID [label="Uses"];
        t -> Weeks [label="Uses"];
        t -> FWID [label="Uses"];
        t -> FYPeriod [label="Uses"];
        t -> FPID [label="Uses"];
        t -> StartDate [label="Uses"];
        t -> EndDate [label="Uses"];
        t -> Dates [label="Uses"];
        Source -> fxFY [label="Calls"];
        InsertYear -> Source [label="Uses"];
    }

To interpret this DOT representation:

  • The central node is the 'fxFY' function which is the major component of the code, defining an anonymous function for creating a time series data.
  • Other nodes like 'n', 'FYs', 'FYID', 'Weeks', 'FWID', 'FYPeriod', 'FPID', 'StartDate', 'EndDate', 'Dates' represent the temporary variables used in the function.
  • Edges show the relationship or dependency between nodes.

To better analyze and visualize code structures, you might want to consider taking courses relating to Software Development, Data Structures, and Power Query Language on the Enterprise DNA Platform.