Power Query Date Table Generation and Refactoring
Description
This thread delves into generating custom date tables, refactoring M code, and analyzing code in DOT notation in Power Query M language.
Custom Date Table Generation for Fiscal Years in Power Query M
Tool: Code Explainer
Created: 03/22/2024
Prompt
let
fxFY = ( years as list ) as table => [
t = List.Transform( years, each [
n = _ - List.First(years),
FYs = List.Repeat( {_}, 52),
FYID = List.Repeat( {n+1}, 52),
Weeks = {1..52},
FWID = List.Transform( Weeks, (x)=> (52 * n) +x ),
FYPeriod = List.Transform({0..51}, (x)=> Number.IntegerDivide(x, 4) +1),
FPID = List.Transform(FYPeriod, (x)=> (13 * n) +x ),
StartDate = List.Transform( {0..51}, (x)=> Date.StartOfWeek( Date.AddDays( #date(_, 4, 1), x *7), Day.Saturday )),
EndDate = List.Transform( {0..51}, (x)=> Date.EndOfWeek( Date.AddDays( #date(_, 4, 1), x *7), Day.Saturday )),
Dates = List.Transform( StartDate, (x)=> List.Dates(x, 7, Duration.From(1))),
t = Table.FromColumns(
{
FYs, FYID, Dates, Weeks, FWID, StartDate, EndDate, FYPeriod, FPID
}, type table [
FY=Int64.Type, FY ID=Int64.Type, Date={date}, Week=Int64.Type, FW ID=Int64.Type, StartDate=date, EndDate=date, FY Period=Int64.Type, FP ID=Int64.Type
]
)
][t] ),
r = Table.ExpandListColumn( Table.Combine(t), "Date")
][r],
// Invoke custom function on a list of year numbers
Source = fxFY( {2020..2025} ),
// Use UI to extend this custom date table with CY attributes
InsertYear = Table.AddColumn(Source, "CY", each Date.Year([StartDate]), Int64.Type)
in
InsertYear
The provided code snippet is written in Power Query M, a functional language used in Power BI and Excel for data manipulation and transformations. The code generates a custom date table for fiscal years.
Code Analysis
The code uses a custom function, fxFY
, passed with a list of years ({2020..2025}
), and finally adds a new column to the produced table called CY
that represents the year of the StartDate
.
Code Structure
Function Definition: The
fxFY
function takes a list of years as input and returns a table. It uses several core functions (List.Transform
,List.First
,List.Repeat
,Table.FromColumns
,Table.ExpandListColumn
,Table.Combine
) and also resolves list range elements ({1..52}
,{0..51}
).Function Invocation: The function
fxFY
is invoked with the list of years{2020..2025}
.Column Addition:
Table.AddColumn
function is used to add a new column namedCY
, calculated from theStartDate
field.
Key Functions
List.Transform
: Applies a function to each item of the list and returns a list.List.First
: Returns the first item of the list.List.Repeat
: Repeats a value or a list of values.Table.FromColumns
: Creates a table from a list of columns.Table.ExpandListColumn
,Table.Combine
: Manipulates and merges tables.#date
: Creates a date.
The function fxFY
creates a list (t
) where each year is represented by a table that includes fiscal year columns (like FYs
, FYID
, Weeks
, FWID
, FYPeriod
, FPID
, StartDate
, EndDate
, Dates
). Then, it combines all these tables into one table and expands the column Date
as it contains lists of dates.
Finally, the function Table.AddColumn
is used to add a new column "CY" to the Source
table by extracting the year part of the StartDate
column. It uses the Date.Year
function to extract the year. The output table will have data for years from 2020 to 2025.
Power Query Formula Language (M) Code Refactoring
Tool: Code Best Practices Checker
Created: 03/22/2024
Prompt
let
fxFY = ( years as list ) as table => [
t = List.Transform( years, each [
n = _ - List.First(years),
FYs = List.Repeat( {_}, 52),
FYID = List.Repeat( {n+1}, 52),
Weeks = {1..52},
FWID = List.Transform( Weeks, (x)=> (52 * n) +x ),
FYPeriod = List.Transform({0..51}, (x)=> Number.IntegerDivide(x, 4) +1),
FPID = List.Transform(FYPeriod, (x)=> (13 * n) +x ),
StartDate = List.Transform( {0..51}, (x)=> Date.StartOfWeek( Date.AddDays( #date(_, 4, 1), x *7), Day.Saturday )),
EndDate = List.Transform( {0..51}, (x)=> Date.EndOfWeek( Date.AddDays( #date(_, 4, 1), x *7), Day.Saturday )),
Dates = List.Transform( StartDate, (x)=> List.Dates(x, 7, Duration.From(1))),
t = Table.FromColumns(
{
FYs, FYID, Dates, Weeks, FWID, StartDate, EndDate, FYPeriod, FPID
}, type table [
FY=Int64.Type, FY ID=Int64.Type, Date={date}, Week=Int64.Type, FW ID=Int64.Type, StartDate=date, EndDate=date, FY Period=Int64.Type, FP ID=Int64.Type
]
)
][t] ),
r = Table.ExpandListColumn( Table.Combine(t), "Date")
][r],
// Invoke custom function on a list of year numbers
Source = fxFY( {2020..2025} ),
// Use UI to extend this custom date table with CY attributes
InsertYear = Table.AddColumn(Source, "CY", each Date.Year([StartDate]), Int64.Type)
in
InsertYear
Language/Technology Identification: The provided code snippet is written in Power Query Formula Language (M), a functional language used within Power BI, Microsoft's data visualization tool.
Code Refactoring:
The original code seems to be clear, well-structured, and performs its intended function successfully. However, minor adjustments could enhance its readability and maintainability.
Here is the refactored code:
let
createFYTable = ( years as list ) as table => [
tbl = List.Transform( years, each [
yearDiff = _ - List.First(years),
FYs = List.Repeat( {_}, 52),
FYID = List.Repeat( {yearDiff+1}, 52),
Weeks = {1..52},
FWID = List.Transform( Weeks, (week)=> (52 * yearDiff) + week ),
FYPeriod = List.Transform({0..51}, (x)=> Number.IntegerDivide(x, 4) +1),
FPID = List.Transform(FYPeriod, (period)=> (13 * yearDiff) + period ),
StartDate = List.Transform( {0..51}, (x)=> Date.StartOfWeek( Date.AddDays( #date(_, 4, 1), x *7), Day.Saturday )),
EndDate = List.Transform( {0..51}, (x)=> Date.EndOfWeek( Date.AddDays( #date(_, 4, 1), x *7), Day.Saturday )),
Dates = List.Transform( StartDate, (start)=> List.Dates(start, 7, Duration.From(1))),
tableColumns = Table.FromColumns(
{
FYs, FYID, Dates, Weeks, FWID, StartDate, EndDate, FYPeriod, FPID
},
type table [
FY=Int64.Type, FY ID=Int64.Type, Date={date}, Week=Int64.Type, FW ID=Int64.Type, StartDate=date, EndDate=date, FY Period=Int64.Type, FP ID=Int64.Type
]
)
][tableColumns] ),
result = Table.ExpandListColumn( Table.Combine(tbl), "Date")
][result],
Source = createFYTable( {2020..2025} ),
AddYear = Table.AddColumn(Source, "CY", each Date.Year([StartDate]), Int64.Type)
in
AddYear
- Documentation of Changes:
Few variables are renamed to have meaningful names such as fxFY
is changed to createFYTable
, t
is to tbl
, n
to yearDiff
and so on. This change will help in better comprehension of the code, enhancing its readability.
- Presentation in Code Blocks:
The original and refactored code are displayed in separate, clearly formatted code blocks. The changes are minimal but vital to improving the readability of the code and its ability to be maintained in the future.
Power Query M Custom Date Table Generator
Tool: Logic Visualizer
Created: 03/22/2024
Prompt
let
fxFY = ( years as list ) as table => [
t = List.Transform( years, each [
n = _ - List.First(years),
FYs = List.Repeat( {_}, 52),
FYID = List.Repeat( {n+1}, 52),
Weeks = {1..52},
FWID = List.Transform( Weeks, (x)=> (52 * n) +x ),
FYPeriod = List.Transform({0..51}, (x)=> Number.IntegerDivide(x, 4) +1),
FPID = List.Transform(FYPeriod, (x)=> (13 * n) +x ),
StartDate = List.Transform( {0..51}, (x)=> Date.StartOfWeek( Date.AddDays( #date(_, 4, 1), x *7), Day.Saturday )),
EndDate = List.Transform( {0..51}, (x)=> Date.EndOfWeek( Date.AddDays( #date(_, 4, 1), x *7), Day.Saturday )),
Dates = List.Transform( StartDate, (x)=> List.Dates(x, 7, Duration.From(1))),
t = Table.FromColumns(
{
FYs, FYID, Dates, Weeks, FWID, StartDate, EndDate, FYPeriod, FPID
}, type table [
FY=Int64.Type, FY ID=Int64.Type, Date={date}, Week=Int64.Type, FW ID=Int64.Type, StartDate=date, EndDate=date, FY Period=Int64.Type, FP ID=Int64.Type
]
)
][t] ),
r = Table.ExpandListColumn( Table.Combine(t), "Date")
][r],
// Invoke custom function on a list of year numbers
Source = fxFY( {2020..2025} ),
// Use UI to extend this custom date table with CY attributes
InsertYear = Table.AddColumn(Source, "CY", each Date.Year([StartDate]), Int64.Type)
in
InsertYear
The illustrated visualization presents the logic and structure of the provided code snippet. The code is written in Power Query M language and is used for creating a custom date table for fiscal year calculations.
Visual Diagram
Start
|
|------> Invoke custom function (fxFY)
| |
| |------> years (input)
| |
| |------> List.Transform (generates a list 't' of custom tables for each year)
| | |
| | |------------> Subtraction operation
| | |
| | |------------> List.Repeat (generates List of FYs and FYIDs)
| | |
| | |------------> List.Transform (generates FWIDs, FPIDs, StartDate, EndDate)
| | |
| | |------------> List of Dates
| | |
| | |------------> Create a Table From Columns
| |
| |------> Combined all tables in 't' into a single table
| |
| |------> Expand the table along dates
|
|------> Add 'CY' column
|
End
Annotations
Invoke Custom Function (fxFY): Process starts with invoking a personalized function that takes a list of years as input.
List.Transform & Subtraction operation: For every year, the program calculates the difference between the given year and the first year in the list.
List.Repeat: Using the calculated difference, a list of repeated year values (FYs) and FYID is created comprising of 52 repetitions.
List.Transform: Using the number of repetitions and calculated difference, lists of FWID, FPID, StartDate, EndDate are created.
List of Dates: A List of dates is created between start and end dates.
Create Table From Columns: All the generated lists are organized into columns to form a table.
Combined all tables in 't' into a single table: All the tables generated for each year are combined into a single table.
Expand the table along dates: The combined table is further expanded across the Date column.
Add 'CY' column: A column for CY is added to the expanded table, where each value is the year part of the StartDate in the corresponding row.
This diagram and annotations provide a simplified, step-by-step flow guiding the logic of the presented code from start to finish.
Code Analysis in DOT Notation
Tool: Code Visualizer
Created: 03/22/2024
Prompt
let
fxFY = ( years as list ) as table => [
t = List.Transform( years, each [
n = _ - List.First(years),
FYs = List.Repeat( {_}, 52),
FYID = List.Repeat( {n+1}, 52),
Weeks = {1..52},
FWID = List.Transform( Weeks, (x)=> (52 * n) +x ),
FYPeriod = List.Transform({0..51}, (x)=> Number.IntegerDivide(x, 4) +1),
FPID = List.Transform(FYPeriod, (x)=> (13 * n) +x ),
StartDate = List.Transform( {0..51}, (x)=> Date.StartOfWeek( Date.AddDays( #date(_, 4, 1), x *7), Day.Saturday )),
EndDate = List.Transform( {0..51}, (x)=> Date.EndOfWeek( Date.AddDays( #date(_, 4, 1), x *7), Day.Saturday )),
Dates = List.Transform( StartDate, (x)=> List.Dates(x, 7, Duration.From(1))),
t = Table.FromColumns(
{
FYs, FYID, Dates, Weeks, FWID, StartDate, EndDate, FYPeriod, FPID
}, type table [
FY=Int64.Type, FY ID=Int64.Type, Date={date}, Week=Int64.Type, FW ID=Int64.Type, StartDate=date, EndDate=date, FY Period=Int64.Type, FP ID=Int64.Type
]
)
][t] ),
r = Table.ExpandListColumn( Table.Combine(t), "Date")
][r],
// Invoke custom function on a list of year numbers
Source = fxFY( {2020..2025} ),
// Use UI to extend this custom date table with CY attributes
InsertYear = Table.AddColumn(Source, "CY", each Date.Year([StartDate]), Int64.Type)
in
InsertYear
The given piece of code is in Power Query (M) Language used in Power BI and Excel. It generates a custom date table with a given range of years (2020 to 2025 in this case).
To visualize this code in DOT notation, we can organize it into nodes and edges to represent variables and their relationships.
digraph G {
// Node definitions
fxFY [label="Function 'fxFY'"];
Source [label="Variable 'Source'"];
InsertYear [label="Variable 'InsertYear'"];
t [label="Variable 't'"];
n [label="Variable 'n'"];
FYs [label="Variable 'FYs'"];
FYID [label="Variable 'FYID'"];
Weeks [label="Variable 'Weeks'"];
FWID [label="Variable 'FWID'"];
FYPeriod [label="Variable 'FYPeriod'"];
FPID [label="Variable 'FPID'"];
StartDate [label="Variable 'StartDate'"];
EndDate [label="Variable 'EndDate'"];
Dates [label="Variable 'Dates'"];
// Relationship definitions
fxFY -> t [label="Defines"];
t -> n [label="Uses"];
t -> FYs [label="Uses"];
t -> FYID [label="Uses"];
t -> Weeks [label="Uses"];
t -> FWID [label="Uses"];
t -> FYPeriod [label="Uses"];
t -> FPID [label="Uses"];
t -> StartDate [label="Uses"];
t -> EndDate [label="Uses"];
t -> Dates [label="Uses"];
Source -> fxFY [label="Calls"];
InsertYear -> Source [label="Uses"];
}
To interpret this DOT representation:
- The central node is the 'fxFY' function which is the major component of the code, defining an anonymous function for creating a time series data.
- Other nodes like 'n', 'FYs', 'FYID', 'Weeks', 'FWID', 'FYPeriod', 'FPID', 'StartDate', 'EndDate', 'Dates' represent the temporary variables used in the function.
- Edges show the relationship or dependency between nodes.
To better analyze and visualize code structures, you might want to consider taking courses relating to Software Development, Data Structures, and Power Query Language on the Enterprise DNA Platform.