Global Azure Bootcamp 2021 – Azure Synapse & Purview plus CI/CD for the BI Developer

This years edition of the Global Azure Bootcamp is scheduled for the end of this week. As in the last years, this event is a global one. The global webpage lists 6 continents, >550 speakers and more than 530 sessions around the globe (https://globalazure.net/)

Global Azure Bootcamp Austria 2021

The Austrian Edition takes place on Friday, 16th April 2021 and following the tradition, Rainer Stropek and team are organizing this event.

I am happy that I can again join the event as a speaker and for sure as an attendee – the agenda looks great (https://globalazure.at/) and is filled with a wide range of different Azure topics.

My colleagues Mario Schnalzenberger, Stefan Denninger and Johannes Lagler-Grüner (cubido & ACP) are also presenting – We provide sessions around industrial IoT, IoT central, Azure Sentinel and Azure Synapse Analytics and Azure Purview.

Reserve the time – the event is free, virtual and definitely worth visiting! 😉

Global Azure Virtual 2021 – Lüdinghausen

In the afternoon, I am virtually visiting Lüdinghausen, Germany for another session under the Global Azure umbrella. This time, it’s DevOps time for the Business Intelligence developer. Often, the continuous integration (CI) aspect is underrepresented in business intelligence projects. Therefore, this session should guide you through some of the options how you can bring a business intelligence project under source control, automagically deploy changes and even publish new Power BI reports into your Power Bi environment.

My session – Continuous Integration for the BI Developer starts at 3pm on the – yeah- Main stage 😉

Registration is free and available here:

Register on Eventbrite

The schedule for the Lüdinghausen event can be found here https://sessionize.com/view/212tcpky/gridsmart

Data News – 2021-04-05

Welcome to another edition of #WolfisDataNews! I took a longer break since my last Data news edition – MVPSummit and well – too much work forced me to pause.. but this week is full again with many announcements – especially the Power BI team.. wow… many new features are available!

Azure Data

• You want to migrate your on-premises data workload to the cloud? But you are not sure how to do that? Have a look at the Azure Database Migration Guides (https://docs.microsoft.com/en-us/data-migration/) – Migrate your SQL Server, Oracle, DB2, Access, MySQL, SAP, PostgreSQL, and many more to the cloud using best practices instructions from Microsoft.

Data Governance

• What is a Data Catalog? Value, Benefits and Feature – https://tdan.com/what-is-a-data-catalog-value-benefits-and-features/27998
• How Data and Analytics Leaders Can Master Governance (by Gartner) – https://www.gartner.com/smarterwithgartner/how-data-and-analytics-leaders-can-master-governance/

Azure Synapse

• How to read Apache Delta Lake format in serverless Synapse SQL (by Jovan Popovic) – https://jocapc.medium.com/how-to-read-apache-delta-lake-format-in-serverless-synapse-sql-44460e4efc96
• Using COPY command with Azure Synapse Analytics SQL Dedicated Pool – https://howtoanalytics.info/2021/03/19/using-copy-command-with-azure-synapse-analytics-sql-dedicated-pool/
• Integration with On-Premises Data Sources in Azure Synapse Analytics – https://howtoanalytics.info/2021/02/17/integration-with-on-premises-data-sources-in-azure-synapse-analytics/

Power BI

• Power BI March 2021 Feature Summary – See what’s new in Power BI Desktop, … https://powerbi.microsoft.com/en-us/blog/power-bi-march-2021-feature-summary/
• Power BI as a superset of Azure Analysis Services (RECOMMENDED READING) – https://powerbi.microsoft.com/en-us/blog/power-bi-as-a-superset-of-azure-analysis-services/
• ODBC support available for Paginated Reports in public preview – https://powerbi.microsoft.com/en-us/blog/odbc-support-available-for-paginated-reports-in-public-preview/
• On-premises data gateway March 2021 update – https://powerbi.microsoft.com/en-us/blog/on-premises-data-gateway-march-2021-update-is-now-available/
• Announcing Visual Data Prep general availability (Diagram view in Power Query Online) – https://powerbi.microsoft.com/en-us/blog/announcing-visual-data-prep-general-availability-diagram-view-in-power-query-online/
• Support of external tools in Power BI Desktop is GA – https://powerbi.microsoft.com/en-us/blog/support-of-external-tools-in-power-bi-desktop-is-ga/
• Guide to enabling your organization to use Power BI in Microsoft Teams – https://powerbi.microsoft.com/en-us/blog/guide-to-enabling-your-organization-to-use-power-bi-in-microsoft-teams/
• Announcing new deployment pipelines capabilities – https://powerbi.microsoft.com/en-us/blog/announcing-new-deployment-pipelines-capabilities
• Power BI Premium per user now generally available for purchase –
  • https://powerbi.microsoft.com/en-us/blog/power-bi-premium-per-user-now-generally-available-for-purchase/
  • What’s New in Dataflows – April 2021 – https://powerbi.microsoft.com/en-us/blog/whats-new-in-dataflows-april-2021/
  • New sharing experience coming soon – https://powerbi.microsoft.com/en-us/blog/new-sharing-experience-coming-soon/

Session Recordings

One of my goals for 2021 is to be more active on Youtube and as a first step I created a Playlist containing some of my available session recordings: https://www.youtube.com/channel/UCLyrxmFd-qmk4-wWwL6w7ww

• If you like the content, please follow for updates. Appreciate your feedback!

(Virtual) Events in the next weeks

Reading is fun, but (virtual) conferences too…

• 18. March 2021 – Join us for the Azure Synapse Launch event – Microsoft Austria and cubido provide you with a webinar full of Synapse and the future of Analytics. Save the date – The registration page will open soon.

Have a good start into the week!

Wolfgang

What’s in the Pipeline for Power BI? Join the Business Applications Launch Event..

Maybe you’ve heard about and read the Power Platform release waves including the planned features for Power BI in the next six months. The current version of this release plan is the 2021 release wave 1 plan which describes all new features releasing from April 2021 to September 2021.

Have a look at the web page here: https://docs.microsoft.com/en-us/power-platform-release-plan/2021wave1/

Change log available: the web page gets frequent updates – features get removed, updated and added! (https://docs.microsoft.com/en-us/power-platform-release-plan/2021wave1/change-history)

The Business Applications Launch Event

As usual, the Power Platform / Biz Apps team plans organizes an event to present the highlights from the current release wave. This one, it’s scheduled for April 6, 201 from 9 to 10:30 AM Pacific time.

source: https://powerbi.microsoft.com/en-us/blog/see-all-the-power-bi-updates-at-the-microsoft-business-applications-launch-event-2/

Register for free – see you there!

Try Azure Synapse for free – Serverless SQL and Apache Spark pools – but watch your Azure subscription type

Azure Synapse is the new analytics service that – it seems like – everyone gives a try and wants to learn about. For testing purposes, the costs can provide a blocker somehow. But today, the Synapse team announced, that for a limited-time there are free quantities offered to test the service.

As of today (2021-03-17, 9pm), there is an offer for Serverless SQL Pools for free quantities of 10TB processed per month until the end of July 2021.

For Apache Spark Pools, there are 120 free vCore-hourse per month available until the end of July 2021.

Sounds great… !? BUT…

Check your Azure subscription type (just to be sure)

The free quantities are only free for certain types of Azure subscriptions – so be aware and check your subscription type before you enjoy the free Synapse stuff.

Qualified subscription types for the free Azure Synapse offer

If you are not sure how and where to find your subscription type -> have a look at this blog post and in the docs.microsoft.com page about Azure Cost Management.

Links:

• Limited-time free quantities offer for Azure Synapse Analytics
• Understand Azure Cost Management (Subscription types)

Data News – 2021-03-15

Welcome to another edition of #WolfisDataNews!

It’s Power BI, Synapse Analytics, Teams and some of my session recordings this week.

Power BI

• Power BI Incremental Refresh Parquet files, without a database – PowerBI Incremental refresh Parquet files, without a Database. – Project Controls blog (datamonkeysite.com)
• VNet Connectivity for Power BI Datasets (Public Preview) – Announcing VNet Connectivity for PBI Datasets (Public Preview) | Microsoft Power BI Blog | Microsoft Power BI
• Power BI Premium Gen 2 Autoscale Preview – Announcing Premium Gen 2 Autoscale Preview | Microsoft Power BI Blog | Microsoft Power BI

Azure Synapse Analytics / Azure Analytics

• Samples for Azure Synapse Analytics – GitHub – Azure-Samples/Synapse: Samples for Azure Synapse Analytics
• Understand the roles required to perform common tasks in Synapse – Understand the roles required to perform common tasks in Synapse – Azure Synapse Analytics | Microsoft Docs
• The Azure Analytics Story – In my opinion a GREAT overview about the Azure Analytics options – The Azure Analytics Story | LinkedIn

Microsoft Teams

• What’s New in Microsoft Teams (MS Ignite 2021 Spring edition) – https://techcommunity.microsoft.com/t5/microsoft-teams-blog/what-s-new-in-microsoft-teams-microsoft-ignite-2021/ba-p/2118226
• Introducing MS Whiteboard for Android and Whiteboard for Microsoft Teams channels and Chat – Introducing Microsoft Whiteboard for Android and Whiteboard for Microsoft Teams channels and chat – Microsoft Tech Community

Session Recordings

One of my goals for 2021 is to be more active on Youtube and as a first step I created a Playlist containing some of my available session recordings: https://www.youtube.com/channel/UCLyrxmFd-qmk4-wWwL6w7ww

• If you like the content, please follow for updates. Appreciate your feedback!

(Virtual) Events in the next weeks

Reading is fun, but (virtual) conferences too…

• 18. March 2021 – Join us for the Azure Synapse Launch event – Microsoft Austria and cubido provide you with a webinar full of Synapse and the future of Analytics. Save the date – The registration page will open soon.

Have a good start into the week!

Wolfgang

Create hierarchical business glossary terms in Azure Purview

During Spring Ignite 2021, many new features for Azure products have been announced. And Azure Purview was included in the list of enhancements.

Today, I want to highlight one of these enhancements – the ability to create hierarchical business glossary terms in the Azure Purview data catalog.

Create a new Top-level term

Let’s log into our Purview account and in Purview Studio, please select the Glossary hub. In there, let’s create a new glossary term.

Create a new glossary term in Azure Purview

My new term is based on the System default term template.

Use the System default template for the term

In the term form, please enter the new term definition. I am going to start with a business glossary term Business Partner and will add two child entries afterwards – Supplier and Customer

Create the top level entry – no parent item is selected

The Business Partner term will be my top level term, therefore I do not select a parent entry. In my example, I keep the term definition at a minimal level. It’s only for the hierarchy stuff today! 😉

Next, Add Child entries

As a next step, please add a new term and select the newly created Business Partner term as parent.

Create a child entry and select the parent

Immediately after you selected the parent term, the formal name of the glossary term changes and it follows the syntax of ParentTermName_NewTermName.

Create the entry and repeat with a new term called Supplier (and also set the parent reference to Business Partner).

Browse your Business Glossary

The new terms are now saved in the business glossary. The List view already provides you with a first look at the hierarchical structure (see the link to the Business Partner term in the Customer term).

The list of glossary terms in the glossary catalog

If you change the view to Table view, the hierarchy is also marked there.

The Table view allows for a hierarchical view of the glossary entries

Another way to browse the term hierarchy is to open the Business Partner term and there the Related tab. The Children list contains the direct successors of this term

The detail page with the list of child items

One Level of Hierarchy is not enough

The last test to perform today was to add another level of hierarchy into our glossary term structure. I added a Preferred Supplier term to the hierarchy as a child of the Supplier term. Select the parent term (in my case its the term on level two) and that’s it..

Add another hierarchy level to the glossary hierarchy

On the detail page of the Preferred Supplier term, the hierarchy of terms is displayed and the hyperlinks can be used to directly switch to the other entries.

Hierarchies can have more than one level

To sum it up

The ability to create hierarchical glossary terms in the Purview data catalog allows a better structuring of your business glossary. The interconnection between these items is nicely integrated in the Purview Studio. In one of my next posts, I will have a look at the usage and assignment of hierarchical glossary terms in the Purview Data Map / Catalog.

Stay curious, catalog your data and enjoy life,

Wolfgang

Data News – 2021-03-01

Welcome to another edition of #WolfisDataNews!

Power BI

• February 2021 Feature Summary – Power BI Desktop Feb 2021 is released and with it a list of new features like enhancements to DirectQuery for PBI datasets, Search Bar, new Field list, a Parquet connector in Power BI Desktop, new SAP Hana and BW documentation, ….. and many more- https://powerbi.microsoft.com/en-us/blog/power-bi-february-2021-feature-summary/
• The On-Premises Data Gateway Feb 2021 update is available. https://powerbi.microsoft.com/en-us/blog/on-premises-data-gateway-february-2021-update-is-now-available/
• Object-Level Security in Power BI (public Preview) – In addition to Row Level Security to hide data rows, you now can hide objects (tables and columns) in your Power BI datasets. It is NOT Premium only, PRO too 😉 https://powerbi.microsoft.com/en-us/blog/object-level-security-ols-now-available-for-public-preview-in-power-bi-premium/
• The Power BI connector for Databricks is GA – https://powerbi.microsoft.com/en-us/blog/announcing-general-availability-ga-the-power-bi-connector-for-databricks/
• Power BI Developer community February 2021 update – Power BI Embedded Gen2 Preview, New client API Documentation, Save Reports to different workspaces, Export to file API enhancements, … https://powerbi.microsoft.com/en-us/blog/power-bi-developer-community-february-2021-update/

(Virtual) Events in the next weeks

Reading is fun, but (virtual) conferences too…

THIS WEEK IS MS IGNITE WEEK – Expect many announcements during the week and some of them in the next #WolfisDataNews

• Microsoft Ignite – March 2-4 2021 – https://myignite.microsoft.com/home
• On-Demand sessions are available: Around the Clock – Azure SQL and Azure Data Factory – https://github.com/microsoft/aroundtheclock
• 18. March 2021 – Join us for the Azure Synapse Launch event – Microsoft Austria and cubido provide you with a webinar full of Synapse and the future of Analytics. Save the date – The registration page will open soon.

Have a good start into the week!

Wolfgang

Azure Purview gets more sources

It’s Microsoft Ignite week and it is like a tradition, that the product groups prepare and release new features around that timeframe. When I was preparing another Purview presentation today, I found out that the list of supported sources in Purview got longer.

For those of you, who watched the first MS Mechanics video about Purview, already know these sources, but as of today (2021-02-28) the sources for

• Azure
• Oracle (Preview)
• SAP ECC (Preview)
• SAP S/4Hana (Preview)
• Teradata (Preview)

are available in my Purview instance.

Azure Purview Sources in my Purview tenant (as of 2021-02-28)

The documentation is still not updated, so some screenshots from my side.

Azure Source

The Azure Source in Purview allows you to register an Azure Management Group, a Subscription or an Azure Resource Group to be included in the scans in Azure Purview.

Source options for the new Azure Source in Purview

In the next configuration step (scan configuration), you can choose which types of resources you want to include in the scans.

Oracle

Unfortunately, I have no Oracle system to test at home, therefore -> screenshot only!

Register a new Oracle Source in Azure Purview

SAP ECC

Unfortunately, I have no SAP ECC system to test at home, therefore -> screenshot only!

Register a new SAP ECC source

SAP S/4Hana

Unfortunately, I have no SAP S/4Hana system to test at home, therefore -> screenshot only!

Register a new SAP S/4Hana source

Teradata

Unfortunately, I have no Teradata system to test at home, therefore -> screenshot only!

Teradata source registration in Azure Purview

I like the fact, that the list of supported source systems in Azure Purview gets longer and longer.

While browsing in the Azure Purview documentation, I found the Purview Connector Overview page. The link is here – have a look to see

a) which sources are currently supported (not updated by the way) and

b) to see if Metadata extraction, full/incremental scans, classification and lineage is supported.

=> https://docs.microsoft.com/en-us/azure/purview/purview-connector-overview

Be curious, catalog your data estate and have fun!

Wolfgang

Data News – 2021-02-22

It’s Monday again and it’s time for #WolfisDataNews again!

Power BI

• Gartner MQ for Analytics and BI Platforms 2021: The new Gartner Magic Quadrant was released in the past week. And Microsoft (with the Power BI platform), well, stayed in the leader quadrant and increased the distance to other vendors in the leader quadrant. Read the reaction from the Power BI team at the Power BI blog: Microsoft named a Leader in the 2021 Gartner Magic Quadrant for Analytics and BI Platforms | Microsoft Power BI Blog | Microsoft Power
• Security is key – The Power BI released a new Power BI security whitepaper and many more security enhancements – Data insights without limit, security without compromise
• .NET Core support for Analysis Services client libraries is GA – If you want to include Analysis Services client libraries in your own applications – the .NET Core support is not GA! .NET Core support for Analysis Services client libraries is GA

Analytics

• One of my highlight features in Azure Synapse Analytics are Shared metadata tables. Charl Roux published an article about Spark Table to Serverless Pool Metadata Sync
• Did you know that there is a limit in Synapse workspaces? Workspace Quota Limit -Unable to create Synapse Workspace due to subscription limit

(Virtual) Events in the next weeks

Reading is fun, but (virtual) conferences too…

• Microsoft Ignite – March 2-4 2021 – https://myignite.microsoft.com/home
• On-Demand sessions are available: Around the Clock – Azure SQL and Azure Data Factory – https://github.com/microsoft/aroundtheclock
• 18. March 2021 – Join us for the Azure Synapse Launch event – Microsoft Austria and cubido provide you with a webinar full of Synapse and the future of Analytics. Save the date – The registration page will open soon.

Have a good start into the week!

Wolfgang

Data News – 2021-02-15

It’s Monday again and it’s time for #WolfisDataNews again!

Power BI

• Power BI Embedded Generation 2 is in preview for Azure subscriber now! https://powerbi.microsoft.com/en-us/blog/announcing-power-bi-embedded-generation-2-preview/
• Michael Kovalsky from the Power BI team published a huuuge collection of best practices rules for Tabular Editor for a better Power BI / SSAS model development – https://powerbi.microsoft.com/en-us/blog/best-practice-rules-to-improve-your-models-performance/

Analytics

• The on-demand recording of Introduction of Lakehouse and SQL Analytics (Databricks and Delta lake) is available – https://databricks.com/p/webinar/intro-to-lakehouse-and-sql-analytics-emea
• Python – What is Python? How and where to start? If you are new to Python and want to improve? have a look at the 44 part free video training – Programming for Python | Python for Beginners – https://www.youtube.com/watch?v=jFCNu1-Xdsw&feature=youtu.be
• Microsoft Learn course – Data Integration at scale with Azure Data Factory or Azure Synapse Pipeline – https://docs.microsoft.com/en-us/learn/paths/data-integration-scale-azure-data-factory/

(Virtual) Events in the next weeks

Reading is fun, but (virtual) conferences too…

• Friday, February 19th, 2021 – Virtual Power Platform Bootcamp Austria – We are finalizing the agenda. Stay tuned for a day full of Power Platform sessions, hands-on and stories from the field. Register here: https://www.eventbrite.de/e/virtual-power-platform-bootcamp-austria-2021-tickets-133239189001
• Microsoft Ignite – March 2-4 2021 – https://myignite.microsoft.com/home
• On-Demand sessions are available: Around the Clock – Azure SQL and Azure Data Factory – https://github.com/microsoft/aroundtheclock

Have a good start into the week!

Wolfgang