Thursday 8 December 2022

Sum totals for Columns redux


Back October I showed how to run a quick sum of a column in Power Query. But I curious to know how I could see this total alongside a count of the number of rows. It turns out this is quite easy because you can include text in your output from a Power Query script. In fact you can even combine these two totals with an explanation as I have done above.

A few points to remember in doing this:
1. If you're creating a variable* in Power Query (i.e. a line in the M code) you don't have to call it right away you can call on it in the final "in" line.

2. You can't minx numbers with text unless you convert them to text first.

So the only code you need really (on top of what you many have already done) is:

#"Counted Rows" = Table.RowCount(#"PreviousRow"),
Total = List.Sum(#"Filtered Rows1"[Users])

in

"Total= "&Text.From(Total)&" from "&Text.From(#"Counted Rows")&" rows"


Friday 28 October 2022

Add extra rows in Power Query dependent on value in cell/column

If you're working in Power Query, and you need to add a whole bunch of blank rows, or rows based on the value from another column. How do you do it?

I went round the houses with this one. There are two key pieces to the puzzle. The first is relatively easy/well known. If you split columns by delimiter you can (under advanced options) do this into new rows rather than new columns.

But how to get from the number in the cell/column to something with which split columns by delimiter could work?

At first I tried putting the cell (actually in my case it was a subtracting one cell from another) as a power of ten, but I couldn't work out how to get it out of 1e+48 mode, to then convert to text and delimit. 

Then I came across Catalin Bombea's post on this forum which I modified. If you add a new column using this formula:

= Table.AddColumn(Source, "NewColumnName", each Text.Repeat("¬",[ValueColumn]-1))

This will give you a string of ¬s as long as the value in the cell/column, for each row in the table (even if it's just 1. 

You then just use split by delimiter and away you go. Only tried it with Power BI but it should work with Excel too.

Thursday 20 October 2022

How do you sum a total for a column in Power Query?


Power Query has a "Count Rows" button, but there's no equivalent button for when you want to quickly get the total of your values column, to see total sales or total users etc.

But there's good news! There are two relatively simple ways to do this. The one-step version for those happy to work in Advanced Editor, and the easier two-step version for those who aren't. Let's take them in reverse order.

The easier, two-step, version
1. Choose "Add Column">"Custom Column" and in the formula box just type "Total" (with quotes)
2. Now chose "Transform">"Group By". In the 1st box select your new column, under "Operation" choose "Sum" and under "Column" choose the column that has the values to be summed. Press OK to get the above! 

The one-step Advanced Editor version
In a way this is even simpler. Just go to Advanced Editor, put a column on the end of the last line of code and press "Enter". On the new line type Total = List.Sum(Source[Users]). Change the very last line from Source to Total and press OK.

Hat tip to John Dalesandro's post Microsoft Power Query for Excel Tips and Tricks, which gave me enough to work the rest out. Should work for Power Query in both Power BI and Excel.

Friday 26 August 2022

Directives vs Commands in AHK/AutoHotKey

I couldn't find a direct answer to this question, so thought, once I'd managed to piece it together I'd post it here for any future searchers.

In general programming a command is a specific type of directive which tells a programme to do a particular task. However, AHK plays by its own rules. 

According to Frankie Bagnardi, in AHK "Directives are processed just before the script starts and allow you to customize AutoHotkey’s behaviour". There are two types. The first is just the bits of code at the top of a script, but they are also anything that starts with a #.

So in AHK directives are the bits of code starting with a # (hashtag/pound sign in the US)

So if you use, say, #IfWinActive partway down your code, the the commands that follow it will be specific to that context, until you turn it off (by using the same directive, with no conditions/parameters afterwards).

Directives are a type of commands which use a # to create a context. Bits of code that aren't functions, expressions, variables, parameters, text etc. are all commands, (e.g. Send, Sleep, Run, MsgBox). Directives are a specific subset of the available commands.

And, for the sake of completion, functions are those bits of code that come with brackets(parentheses). Expressions are bits of maths.

- If I've got this wrong, please let me know in the comments without shouting or calling me an idiot...

Saturday 20 August 2022

iPad Native Keyboard Shortcuts


I've had an iPad for over a decade and always found it's keyboard, particularly the lack of backwards and forwards keys, frustrating. Recently however I got a new model and, quite by accident I discovered a useful shortcut that when I Googled, no-one else seems to have picked up on (or all the people who have are buried below the usual pile of SEO-d crap).

Anyway if you click the "I" key and then the "O" key in quick succession it selects the word the cursor is on or after. 

IO, just like like that!

In other words I-O (I+O?) is like double clicking with a mouse. It's a minor thing but gets me wondering what other hidden combos there are.

I set off trying to find out and then discovered that Apple have, in fact, done something closer to left and right arrow buttons. If you press and hold the space bar the markings blank out and you can then use the keyboard as a track pack. I got that thanks to this video by Steve Katz.

Does anyone know of any others?

Wednesday 3 August 2022

How to get an editable list of all the files in a folder from Windows Explorer

Often you have a lot of files in a folder and for some reason you want to paste them in Word, Excel, Notepad or something else, but a straight select all, copy & paste doesn't work. While solutions exist for that online using the registry editor, they sound a bit foreboding or complicated, or your IT won't et you have permission. 

Here's an easier way: Copy the folder path from the bar in Windows Explorer (e.g. C:/Users/your.name/Documents) and drop it in to the URL bar in MS Edge or Chrome. Et voila! You can then copy and paste the text wherever you want.

If this doesn't work either check you've used "/"s and not "\"s. Secondly try using a different folder and navigating your way there through the folders in Chrome/Edge. 


Friday 1 July 2022

How to Differentiate Between Similar Field Entries


Say you run a series of events and want to display registration, attendance etc on a bar chart in Power BI. The length of the event names and limitations on the Power BI naming fields in bar charts mean that you can't use the name on the Y-axis, and date gives you problems as well. So you go for a variation on the date worked out in a calculated field during the Power Query part of the process.

This is all fine until the day you run two events on the same day. Now Power BI thinks this is just one event. So you need to find someway of telling the difference between the two events, preferably with a reference to the date of the event.

A way to do this is to add a further calculated column using DAX. This will append an a & b (etc.)  onto the date field when there's a clash, but be blank otherwise. Here's the DAX to do that where 

St.Date =
VAR distinctlycounting = COUNTROWS(FILTER(Titles,Titles[Date]=EARLIER(Titles[Date])))
VAR ReIndex = CALCULATE(DISTINCTCOUNT(Titles[Title]),FILTER(Titles,
Titles[Date]=EARLIER(Titles[Date])&&
Titles[Title]<=EARLIER(Titles[Title])))

RETURN IF(distinctlycounting = 1,Titles[Start Date],SWITCH(ReIndex,
1, [Start Date] & "a",
2, [Start Date] & "b",
3, [Start Date] & "c",
4, [Start Date] & "d",
5, [Start Date] & "e",
6, [Start Date] & "f",
7, [Start Date] & "g",
8, [Start Date] & "h"
))

Where the table is called "Titles", the date field is called "Date" and "Start Date" is the calculated field in Power Query.

Obviously you can add more letter variations if you have more events in a single day, or do away with that bit altogether if you just want to use numbers (and you can also get rid of the CONVERT function wrap around as well.

Wednesday 25 May 2022

Joining/Merging Columns in a single step


As per my previous post sometimes, for tidiness you just want to do something in a single step. So how do you do this with merging / joining / concatenating two columns into one in a single step in Power Query. Easy! (when you know how). Just paste this into your Advanced Editor:

 #"Merged Columns" = Table.CombineColumns(Table.TransformColumnTypes(#"Previous Step Name", {{"Column2", type text}}, "en-GB"),{"Column1", "Column2"},Combiner.CombineTextByDelimiter("", QuoteStyle.None),"Column1"),
Where "Previous Step Name" is the name of the previous step, and "Column1" and "Column 2" are the imaginative titles for the two columns you want to merge. This will leave you with a single column called Column 1.

Remove/Replacing part of a string in Power Query


If you like keeping things neat and understandable in Power BI then you probably like to minimise things to one line. So there are times when, yeah, you could create a new IF column and then delete the original and rename it, but you're probably asking "how can I do this in one line of M code?"

Here's a solution for when you're trying to either remove a certain string(s) within a field, (as well as links to one that shows how to replace them).

Remove part of a string in a field
Say I have a field like this:

My Column
Which platforms do you use? Select all that apply
Tick those which work for you. Select all that apply
Which would you recommend? Select all that apply
Which are value for money? Select all that apply
Select all that apply

I want to get rid of all those uses of "Select all that apply". If I use the standard "replace" tool in Power Query it will blank the fifth line and leave the others untouched. I could create a new column here and then delete the original and rename the new one, but to do it in one move I can use this formula in the Advanced Editor:

#"Replaced Value" = Table.TransformColumns(#"Previous Step Name",{{"My Column", each List.Accumulate({{"Select all that apply","XX"}},_,(string,replace) => Text.Replace(string,replace{0},replace{1}))}}),

Where "XX" is the term you are replacing it with (in this case just put "" i.e. blank).* 

It turns out you can also hit several replace terms in one columns at once by adding them into the second set of double curly brackets. So, if I want to replace xx with XX, yy with YY and zz with ZZ in Column1, I can do it like this

#"Replaced Value" = Table.TransformColumns(#"Previous Step Name",{{"Column1", each List.Accumulate({{"xx","XX"},{"yy","YY"},{"zz","ZZ"}},_,(string,replace) => Text.Replace(string,replace{0},replace{1}))}}),

Alternatively if you want to hit multiple fields you can do that too.  So, if I want to replace xx with XX in column1, and bb with BB in Column2 I can write:

#"Replaced Value" = Table.TransformColumns(#"Previous Step Name",{
{"Column1", each List.Accumulate({{"xx","XX"}},_,(string,replace) => Text.Replace(string,replace{0},replace{1}))},
{"Column2", each List.Accumulate({{"bb","BB"}},_,(string,replace) => Text.Replace(string,replace{0},replace{1}))}
}),

And obviously you can do both together if you wish.


*I found this info in the comments to the two videos included in this blog post by Guru G. The blog post itself covers two similar operations, 1. Removing multiple single characters from a field (video) & 2. Replacing a strings within a field with another string (video). 

Edit
Here's another simpler way from The Biccountant which is closer to the code you get if you use the standard tool.

= Table.ReplaceValue(#"Previous Step Name", each [Text to remove], "" ,Replacer.ReplaceText,{"Field to remove it from"})
Where the field containing the info you want to remove is called [Text to remove] and the field you are removing it from is called [Field to remove it from].

If you want to replace it with a specific value you just write a word or phrase in text in double quotes.
Alternatively, rather than a blank or text you can replace with the contents of another field, you just add this into the 3rd term like so:
= Table.ReplaceValue(#"Previous Step Name", each [Text to remove], each [Text to replace with], Replacer.ReplaceText,{"Field to remove it from"})
Where [Text to replace with] is the new field you want to replace with.

Friday 1 April 2022

Regex Differs Between Google Analytics and Google Search Console

If you're working with both Google Analytics and Google Search Console you've probably tried at some stage to get your head around regex. Regex is brilliant and is well worth investing time into getting to know. It's used by numerous platforms, but does vary a little from platform to platform.

However, Google takes things a step further. Regex is different on Google Analytics than it is in Google Search Console. 

This is particularly noticeable if you're trying to filter your search by URL and want to use markers to tie your needle to either the start or the end of the URL. Ordinarily you use a caret ^ to tie your search term to the start of the available text (or line), and a dollar sign $ to tie to the end. But Google complicates this.

So for Google Analytics if you want to find only those pages ending in your search term you use the $. If you want to make sure that this comes straight off the homepage you only need to include the caret - no need to refer to the stem. So if your domain is https://www.datageek.com using
^/news/$
Will find solely https://www.datageek.com/news/, no sub pages and not others like https://www.datageek.com/someothersection/news/

To do the same in Google Search Console though seems to want you to account for the variations in the stem, So to do the same as the above in Search Console you have to use
h.*datageek\.com/news/$
You would think Google would just choose one, but apparently not...