Posts

Local Authority Shape Map in Power BI

Image
 This post can now be found here Thought it was about time to get in shape This is a bit of a tangent but wanted to share useful links as this is something that I have pulled together with multiple sources and not something that I could find a joined view of. So maps are tricky. Getting your data in the right format, right place and most difficult correct shape is really difficult. One approach is to focus only on getting your data in the right place by using latitude and longitude to plot data in the correct places but shape maps are definitely the best way to present data about geographies such as Countries, Regions, Local Authorities, Postcode Areas etc. Out of the box there are a few shape maps like US states, UK countries but for my purposes I need shape maps for Local Authorities, Postcode Areas, Regions and other geographic distributions inside the UK. Custom maps can be loaded and as this article says:  "You can use custom maps with Shape Map as long as they are in th...
Post a Comment
Read more

VADER and Power BI

Image
 This post can now be found here "Search your feelings. you know it to be true" Valence Aware Dictionary and sEntiment Reasoner ( VADER ) is the main alternative to TextBlob and is designed to work with social media text primarily. Again this has been a really simple adaptation of the code used so far in all of the text analytics tools we have applied . The outputs of VADER are a little different to TextBlob and does not include scoring for subjectivity but does break down into 4 aspects the Compound Score which is the key sentiment scale but also individual scores for pos: positive, neu: neural and neg:negative that are ratios of the proportion of the text analysed so you can drill into a bit more context. TextBlob in Power BI import pandas as pd from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer analyser = SentimentIntensityAnalyzer() data = [] for idx, row in dataset.iterrows(): if not isinstance(row['text'], str): continue s...
Post a Comment
Read more

TextBlob and Power BI

Image
 This post can now be found here TextBlob makes me polarity:1  Expanding text capabilities with Sentiment in Power BI is one of the key things I have always wanted to be able to do. The code for this is pretty much the same as covered in previously and as with all of the tools I have covered so far this is the most basic application of each tool - there is a lot more that can  be done with further customisation based on the types of data you are working with. TextBlob is a Natural Language Processing tool (NLP) to identify the sentiment (polarity) of a text string on a scale of -1 (negative) to 1 (positive). As well as sentiment you can export a subjectivity score on a scale of 0 (objective) to 1 (subjective) which can be a really useful way of exploring your and the correlation between sentiment and subjectivity. TextBlob in Power BI import pandas as pd import textblob from textblob import TextBlob data = [] #create a for loop of the rows in the df dataframe for id...
Post a Comment
Read more

Rapid Automatic Keyword Extraction (RAKE) in Power BI

Image
 This post can now be found here I like Python but it needs more acronyms In this post I am going to go through the basic use of Rapid Automatic Keyword Extraction (RAKE) using the Natural Language Tool Kit (NLTK) as documented here .  So before we get into RAKE what is NLTK? It is a set of tools that do some quite similar stuff to Natural Language Processing as used in the spaCy Position of Speech (POS) and Named Entity Recognition (NER) tools that are explored here . Essentially the tools break down text into their component parts for scripts like RAKE, POS, NER to return results based on analysis of those components. What is RAKE? This tool extracts key words and phrases based on frequency and without any understanding of the context of the text being analysed. This makes it quite a general tool that will not work well with all text sources but like POS and NER is a good exploratory tool and may work well when joined up to other information about the text data. As an exampl...
Post a Comment
Read more

Natural Language Processing in Power BI

Image
This post can now be found here With great Power BI comes great responsibility It is time: Power BI + Python = Amazing!  It worked, it is relatively easy to convert the code to run within Power BI. The main thing that changes is the emphasis of using print() or exporting data in some ways things are easier in that you just need to update or create data frames so 🐼s is key... Pandas . In terms of a step by step guide the first thing I did was get the code working in Jupyter as a proof that the code will execute successfully.  Following on from using spaCy . When you add a Python Script to a Power BI Query it basically takes the data as it stands in the previous step and converts it to a data frame called dataset. This means that for the code I had for spaCy the name "df" needed to be changed to "dataset". The other thing that needs to be different is what you do with the outputs of your code. I previously was printing out the values to test that it was working whic...
Post a Comment
Read more

Lost in spaCy

Image
This post can now be found here In spaCy no one can hear you scream... when you get a Syntax error. Experiments with spaCy are going well in the appropriately named Jupyter notebooks. There are two spaCy Natural Language Processing (NLP) components I am working with 1. Position of Speech (POS) and 2. Named Entity Recognition  This tutorial  was really helpful in generalising the experiments I have been doing and looking at different aspects and capabilities of NLP. The idea with this is to get a general set of tools developed in a notebook before looking at adapting the code to be used in Power BI to run NLP as a step in Power Query. The big question is if you can load the NLP language library into Python running in Power BI - if not it is going to be tricky. Starting Point So I am starting from an example here where NLP tools have been used to get an overview count of the contents of a csv of text values - which is great - but not what I need for a Power BI report to be able...
Post a Comment
Read more

Starting Py-Fun

Image
This post can now be found here   💡The Idea I wanted a way to document and encourage practicing of my experiments with Python and integrating it with Azure ML, Power BI which is a major project I am currently working on. 📋The Plan I am going to blog as regularly as possible covering the Python exploration and investigation that I am doing. The plan is to help document the challenges and experience of trying to work to bring Python into the Microsoft Power Platform and processes that are used in my team. This area is something that I have not found many resources about in my work so far so I am conscious that some of this may be of help to others which is another reason to put this into a blog. 👣Next Steps 1) Publish this post 2) Start looking at Natural Language Processing in Python (using Jupyter notebooks) 3) Convert Jupyter code to Power BI for use in data models
Post a Comment
Read more