Thread

Data Analysis and Time Series Breakdown

Explore Change Point Analysis and DAX Measures for identifying consecutive positive days in time series data.

Empty image or helper icon

Data Analysis and Time Series Breakdown

Description

Explore Change Point Analysis and DAX Measures for identifying consecutive positive days in time series data.

Change Point Analysis for Time Series Data

Tool: Languages Advisor

Created: 03/22/2024

Prompt

Two files attached. I posted a topic over on Microsoft, but I donā€™t have the ability to attach files, so Iā€™m redirecting the question here. I wonā€™t rewrite the post, but Iā€™ll give you a summary.

Iā€™m trying to create a measure that will count the number of consecutive days a column value is positive or negative. Iā€™m made some incremental progress, but havenā€™t been able to get a successful outcome.

Iā€™m trying to create one measure: Positive/Negative Consecutive Days Count. But Iā€™m willing to have them as separate measures, as: Positive Consecutive Days Count and Negative Consecutive Days Count.

Sure, to solve your problem, you can leverage the concept of "Change Point Analysis", wherein the idea is to identify points at which the data changes in a prolonged and significant way.

The first thing you should do is create two calculated columns, one for negative values and another for positive values. You should set a value of 1 if the value is negative, or positive depending on the column and null or blank to the counterpart.

DAX code for new calculated columns

PositiveValueFlag = 
IF( [Your Column] > 0, 1, BLANK() )

NegativeValueFlag = 
IF( [Your Column] < 0, 1, BLANK() )

Then, a calculated measure can be created for both positive and negative that identifies when the value changes from positive to negative (or vice-versa).

DAX code for measures

PositiveConsecutiveDaysCount = 
   VAR CurrentDate = MAX( YourTable[Date] )
   VAR CurrentPosFlag = MAX( YourTable[PositiveValueFlag] )
   RETURN
   IF( 
     CurrentPosFlag = 1, 
     CALCULATE(
         COUNT( YourTable[Date] ), 
         FILTER(
           ALL( YourTable ),
           YourTable[Date] <= CurrentDate && YourTable[PositiveValueFlag] = CurrentPosFlag
         ),
         ALLEXCEPT( YourTable, YourTable[PositiveValueFlag] )
     )
)

NegativeConsecutiveDaysCount = 
   VAR CurrentDate = MAX( YourTable[Date] )
   VAR CurrentNegFlag = MAX( YourTable[NegativeValueFlag] )
   RETURN
   IF( 
     CurrentNegFlag = 1, 
     CALCULATE(
         COUNT( YourTable[Date] ), 
         FILTER(
           ALL( YourTable ),
           YourTable[Date] <= CurrentDate && YourTable[NegativeValueFlag] = CurrentNegFlag
         ),
         ALLEXCEPT( YourTable, YourTable[NegativeValueFlag] )
     )
)

Please replace 'YourTable' and 'Your Column' with your actual table and column names.

These measures calculate a running total of consecutive positive and negative values, respectively.

As for further refining your skills in DAX and Power BI, I'd recommend enrolling in related courses on the Enterprise DNA Platform. The best way to learn is through immersive and practical lessons that guide you through real-world applications of DAX in data visualization and analytics.

DAX Consecutive Positive Days Counter

Tool: Code Explainer

Created: 03/22/2024

Prompt

PositiveConsecutiveDaysCount = 
   VAR CurrentDate = MAX( YourTable[Date] )
   VAR CurrentPosFlag = MAX( YourTable[PositiveValueFlag] )
   RETURN
   IF( 
     CurrentPosFlag = 1, 
     CALCULATE(
         COUNT( YourTable[Date] ), 
         FILTER(
           ALL( YourTable ),
           YourTable[Date] <= CurrentDate && YourTable[PositiveValueFlag] = CurrentPosFlag
         ),
         ALLEXCEPT( YourTable, YourTable[PositiveValueFlag] )
     )
)

The code snippet you provided is written in Data Analysis Expressions (DAX), a high-performance formula language introduced by Microsoft for handling data manipulation and analysis within Power BI, Analysis Services, and Power Pivot in Excel.

Let's dissect the provided DAX formula:

Overall:

The goal of this entire expression is to calculate the count of consecutive positive days i.e., tally up the number of successive dates where 'PositiveValueFlag' is 1 (indicating positivity).

Code Analysis:

  1. VAR CurrentDate = MAX( YourTable[Date] ): This line initiates a variable named 'CurrentDate'. The MAX function is used to find the most recent (or maximum) date in the 'Date' column of 'YourTable'.

  2. VAR CurrentPosFlag = MAX( YourTable[PositiveValueFlag] ): This initiates another variable called 'CurrentPosFlag'. It gets the maximum value from the 'PositiveValueFlag' column of 'YourTable'.

  3. RETURN IF( CurrentPosFlag = 1: This starts the main calculation of the formula. If the 'CurrentPosFlag' is equal to 1, it proceeds with the calculation.

  4. CALCULATE( COUNT( YourTable[Date] )): The CALCULATE function changes the context in which the COUNT function works. Here, it will count the number of rows of 'YourTable' where the date is lesser than or equal to the 'CurrentDate' and 'PositiveValueFlag' is equal to the 'CurrentPosFlag' (which is 1, as per the earlier condition).

  5. FILTER( ALL( YourTable ), YourTable[Date] <= CurrentDate && YourTable[PositiveValueFlag] = CurrentPosFlag)): The FILTER function essentially generates a new table with the filtered conditions.

  6. ALLEXCEPT( YourTable, YourTable[PositiveValueFlag] )): The ALLEXCEPT function takes 'YourTable' as a base table and excludes filter context only from 'PositiveValueFlag'.

Key Concept:

One key concept used here is CALCULATE, an important function in DAX as it influences the context in which calculations are performed (known as context transition).

Additional Example:

If we want to get the count of negative consecutive days, we can modify the code like this:

NegativeConsecutiveDaysCount = 
   VAR CurrentDate = MAX( YourTable[Date] )
   VAR CurrentNegFlag = MIN( YourTable[NegativeValueFlag] )
   RETURN
   IF( 
     CurrentNegFlag = 1, 
     CALCULATE(
         COUNT( YourTable[Date] ), 
         FILTER(
           ALL( YourTable ),
           YourTable[Date] <= CurrentDate && YourTable[NegativeValueFlag] = CurrentNegFlag
         ),
         ALLEXCEPT( YourTable, YourTable[NegativeValueFlag] )
     )
)

This example does the same thing as the initial code, but for the count of consecutive negative days.

PositiveConsecutiveDaysCounter

Tool: Documentation Generator

Created: 03/22/2024

Prompt

PositiveConsecutiveDaysCount = 
   VAR CurrentDate = MAX( YourTable[Date] )
   VAR CurrentPosFlag = MAX( YourTable[PositiveValueFlag] )
   RETURN
   IF( 
     CurrentPosFlag = 1, 
     CALCULATE(
         COUNT( YourTable[Date] ), 
         FILTER(
           ALL( YourTable ),
           YourTable[Date] <= CurrentDate && YourTable[PositiveValueFlag] = CurrentPosFlag
         ),
         ALLEXCEPT( YourTable, YourTable[PositiveValueFlag] )
     )
)

PositiveConsecutiveDaysCount Code Documentation

This DAX (Data Analysis Expressions) implementation focuses on calculating the count of consecutive days where a certain value (flag) is positive in a given dataset. We can ensure optimal use of this code in analytics and report generation involving time-series and flag-based data.

Below, we explore how this works:

PositiveConsecutiveDaysCount = 
   VAR CurrentDate = MAX( YourTable[Date] ) // Define CurrentDate as the most recent date in YourTable
   VAR CurrentPosFlag = MAX( YourTable[PositiveValueFlag] ) // Define CurrentPosFlag as the maximum value of PositiveValueFlag in YourTable
   RETURN
   IF( 
     CurrentPosFlag = 1,  // If the CurrentPosFlag equals 1
     CALCULATE(
         COUNT( YourTable[Date] ),  // Count all rows in YourTable where below conditions apply
         FILTER(
           ALL( YourTable ), // Start filtering on entire YourTable
           YourTable[Date] <= CurrentDate && YourTable[PositiveValueFlag] = CurrentPosFlag // include rows where Date is lesser or equal to CurrentDate and PositiveValueFlag equals to CurrentPosFlag
         ),
         ALLEXCEPT( YourTable, YourTable[PositiveValueFlag] ) // Keep all values in YourTable, except for changes in PositiveValueFlag
     )
)

Analyzing the Code

This DAX block calculates PositiveConsecutiveDaysCount. It starts by assigning the latest date and max PositiveValueFlag from YourTable to CurrentDate and CurrentPosFlag respectively.

The method then makes an IF condition check: if CurrentPosFlag equals 1, it makes a COUNT operation on YourTable[Date], filtered by the conditions that YourTable[Date] is less than or equal to CurrentDate and YourTable[PositiveValueFlag] equals CurrentPosFlag.

ALL function defeats any filters that might be affecting YourTable, essentially it's a way to guarantee that you're evaluating the whole column without restrictions. ALLEXCEPT is used to keep all original row contexts from YourTable, but while ignoring any row context from PositiveValueFlag.

The result of this operation gives you the count of consecutive days where a PositiveValueFlag is true. This could be useful for situations where we need to analyze patterns or make comparisons.

A full understanding of the tables and fields in use, as well as the rules around PositiveValueFlag and its calculation, would provide a more detailed understanding of this DAX block. It is also advised to have substantial knowledge about the DAX language before making any modifications. You can learn more from the Enterprise DNA Platform.

Usage

This block of code can be adapted for similar tasks where the objective is to count rows based on conditions and without being affected by any other filter context. You just need to replace YourTable, Date and PositiveValueFlag with your own table and columns respectively.