One may ask: Why Wolfgang, why no blog post about that feature? Well, the answer is easy: it’s in preview and the preview limitations are strict – the enforcement side of the policies are limited certain Azure regions. And therefore, I am planning to try it (and blog about it) when it will be available in more regions.
With that, the root collection and its assigned administrators get more and more important. The permissions in Azure Purview are configured in Purview Studio (Collections -> select a collection -> Role assignments).
Azure Purview role assignments in Purview studio
What I found out today, is that you can add a Purview Root Collection Admin directly in the Azure Portal. Open you Purview account, and select the Root collection permission menu entry.
Purview Root Collection Admin
Select one member account and add it to the list of collection administrator.
Successfully added an additional root collection admin
One section I really like and definitely will have a closer look at is the Implementation Success. It guides you through the process of setting up the ground for a successful POC project. Even after going through several Synapse POCs & projects, I already spotted some additional things to consider.
The content is provided and maintained by the Azure Synapse Customer Success Engineering (CSE) group.
I am happy. More than happy that I can share the news now…
Next March, in London, I am allowed to conduct a full day training at SQLBits 2022. About one of my highlight topics in the past months – Azure Purview and Data Governance.
The list of Training days is already published, registration will open early next year. If you are not sure if the content of the workshop titled Data Governance in a Nutshell – From Theory to Hands-On with Azure Purview is suitable for you, here are some details:
Data is important for every organization. To know your data estate (systems), their dependencies, the responsible people and processes takes your company’s data literacy to the next level. This topic is called Data Governance, which includes two main streams: a) the organizational processes and policies and the b) the technical tools to support the organizational processes.
Many of us think, that Data Governance is a technical problem to solve, but I would start the Data Governance day with an explanation what DG is. We will talk about the people involved, the processes needed, policies that should be in place and the technology so support the processes. The overall Data Governance vision and strategy will also be discussed.
After the DG theory part, we will dive into a Data Governance tool – Azure Purview. In the Hands-On lab we will connect to data sources, configure scans, see how Purview classifies data, assign and work with a business glossary, work with & analyze data lineage and see the Purview data catalog in action. In the last part of the day, we will see Purview integration in action – the Apache ATLAS API will be presented as well the integration into Azure Data Factory and Azure Synapse Analytics.
At the end of the day, attendees should have a feeling about what Data Governance is, the organizational requirements as well as a first look into Azure Purview and how a technical answer to these questions can look like.
In short: the day will guide you through the foundation patterns of Data Governance, the way how Azure Purview can solve (some) of the challenges for Data Governance and get you an idea how you and your organization can start/move further on with Data Governance.
As a follow up of todays PASSDataCommunitySummit keynote, in which Arun Ulag mentioned a new phrase – the Complete Analytics Fabric, I would like to write more about a new option to really get a complete picture of your Azure Synapse Analytics artifacts in your Synapse workspace in one got – in one Power BI dataset – with one connection option only. Sounds interesting? Yes.. it is a nice way..
The new Azure Synapse Analytics Workspace Connector for Power Query
The November Release of Power BI Desktop includes a preview version of this new connector (Nov 2021 release blog post)
Let’s see how this connector works and which options you get by using it. In my demo example, I am using Power BI Desktop and as a first action, use the Get Data dialog to connect to a Synapse Workspace.
There are (as of today) two options to connect to Synapse Analytics – the “old” Azure Synapse Analytics (SQL DW) connector and the new Synapse Workspace connector. Select the new one and move on to the next step..
The new Azure Synapse Analytics workspace connector
Which is a warning that this is a development connector and third-party software is used to connect. I’ll continue …
The warning when using the connector the first time
Up next – Authenticate using your AAD user that has access to a Synapse Analytics workspace.
The next dialog lists all the Synapse workspace your user has access to
The list of Synapse workspace the authenticated user has access to
Whenever I expand one of the workspace nodes, the available data artifacts (databases) are listed. In my demo workspace, the following objects are available. The question that comes up now is: What are all these objects? SQL Dedicated Pools? Serverless databases? Spark databases?
The data artifacts in my Synapse Workspace
If we compare the list in Power BI Desktop and the objects in Synapse Studio, it will get clearer which objects are now available in the Power Query connection dialog.
Which objects are available in the Power Query connector?
As of today, Spark databases (now called Lake databases in Synapse Studio), SQL Serverless databases and Dedicated Pools are available for a connection.
The Synapse Analytics workspace connector now allows you to select tables (objects) from different sources and them in one connection and data access action.
Select different tables and use them in Power BI
In my example, Power BI did not ask if I want an Import or Direct Query connection – Import mode was chosen for me.
In my opinion, this new connector really makes our life as Synapse and/or Power BI developer easier – only one connector to use them all. Let’s see if the list of supported artifacts is extended (like it is mentioned in the announcement blog post) – this really would be very nice: SQL Pools, Spark databases and data lakes, and even linked services connected in one go.
Some months ago I switched to Microsoft Edge as my default web browser. I don’t know when it started, but out of a sudden the copy&paste of URLs out of Edge ended strange in the destination application. A new feature in Edge changed the format of copied URLs in a more “user-friendly” format. Which is nice, yes, .. but…
User-Friendly Formatting for copied URLs
Just in case you do not know how this user-friendly URL does look like:
User-friendly URL copy & paste
How to change it back to URL only
For my job, I often have to write documentations and other documents and in those I want to see the real URL. So I started to do a little research and there are several options to the back to the URL only:
Change the Paste mode in Word (Keep as Text only)
Change the default way, how Edge copies URL
This is my preferred way – change the default setting how MS Edge copies URLs. Short way: open Edge and enter edge://settings/shareCopyPaste into your address bar (or you open Edge settings and navigate to the Share, copy and paste section).
This opens the settings page for URL sharing. Choose the link option if you want the user-friendly information, choose the plain text version if you want the link only.
I hope you liked this tip – otherwise this post is another reference for future Wolfi 😉
Bob Ward demos (and Connor joined 🙂 the “it just gets faster without code changes” in SQL Server 2022 (Parameter sensitive plan optimization)
Bob and Connor discussing 🙂No code change required – only the compatibility level of the database is changed.. and the magic happens
More about the (Azure) SQL Family
The (Azure) SQL family
“Build once – deploy anywhere”
Azure Arc-enabled data services
Azure Cosmos DB
Announcement: Azure Managed Instance for Apache Cassandra
Azure Synapse Link for SQL Server (2022+)
The Azure Synapse Link family is extended to SQL Server (2022+) and will be available in the future. Synapse Link is more than “just a linked server” -> Synapse Link publishes changes to Synapse and can then be used for analytics in Power BI.
Azure Synapse Link for Dataverse
It’s GA. Do some of you remember the Export to DataLake functionality? yes.. Synapse Link for Dataverse is definitely MORE than that…
From Dataverse to Azure Synapse to your the reporting tool of your choice.
Data is pushed from Dataverse to the landing part in Azure Synapse. A Spark database (containing external table definitions to the linked tables) is created. You can use this tables in either Spark or SQL Serverless (using the shared metadata approach) ..
Intelligence Platform
Arun takes over the stage
Empower every individual / every team / every organization
Power BI + Teams integration
Power BI + Excel integration
Power BI Goals
The Complete Analytics Fabric => Power BI and Synapse
Announcements for Power BI
Patrick talks about Hybrid Tables and Automatic Aggregations.
Analyze the effects of Automatic AggregationsHybrid Tables – Import for historical data + Direct Query for the most up-to-date data.
Data Lineage is one of the key features required for a good understanding of your data landscape. Btw. if you want to learn more about data lineage, g2.com has a very good introduction: https://learn.g2.com/data-lineage
This week is Microsoft Ignite 2021 week – virtual again! Three days full of content, mostly pre-recorded sessions and many announcements of new features.
Microsoft published – like for some other conferences in the past – a Book of News, which contains an overview about the features, products and options announced during the conference.
Those of you that know me, maybe 🙂 know my favorite topics – Azure Data, Power BI and Azure Purview. If you have a look at the Book of News, the section for Azure Data is not that long, BUT…
The section of Azure Data announcements at Microsoft Ignite 2021
The Azure Synapse team announced some features around a third, new type of analytic pools – Data Explorer pools. With this type of analytics runtime, we are able to query log and telemetry data.
Azure Synapse Link for Dataverse is now GA. The concept of Synapse Link already an ETL-free extension of your analytic workspace into/ Cosmos DB and now is extended to support Dataverse. With this connection, selected tables out of a Dataverse database are replicated into a Synapse Workspace and are there exposed in a Spark database (which is – through shared metadata – accessible by the SQL Serverless runtime).
One of my overall highlights – Database templates – does not sound really interesting at the beginning. BUT – What I found out so far and if I understood it correctly, these database templates allow for a creation of a lake database based on data lake files. WITHOUT the need for coding – a UI-based editor is included in the first preview version. I am very curious to find out more about these lake databases.
Delta Lake support is now GA. The functionality of read-access to delta lake files through the SQL Serverless pool was already available in preview for some time, now this functionality got into GA state.
As you might have noticed, Azure Purview is one of my newest friends in Azure Data town. During Ignite, the support for Amazon RDS (Relational Database Service), the Data Lake Data Asset Access Governance, and Microsoft Defender for Cloud Integration with Azure Purview was announced.
What I really look forward to test is the Data Asset Access Governance for Data Lake storages. Imagine a world that allows you to define permission on a central place and those permissions are brought to a storage account / system of your choice..
Power BI and Teams are getting better together – the Power BI app for Teams is now GA and allows a deep integration of your Power BI environment directly in your teams application.
Power BI Premium Gen2 is now GA – allowing for a more enterprise grade and scalable analytics.
The concept of Hybrid tables allows a dynamic combination of in-memory caches of historical data combined with real-time data queried in DirectQuery mode.
Already featured in the Azure Purview launch event, the Power BI integration is now GA, allowing for a deeper integration and metadata scanning of your Power BI workspaces, datasets, reports and even dataset structures.
Although the Book of News section is not that impressive, a closer look at the new feature really makes me look forward to a bright future of Azure Data. One of my highlights are the announced Database templates and Lake database in Azure Synapse…
Last week was a special one – I went to a conference, a real conference. 🙂
DirectionsEMEA 2021 took place at Milan from Wednesday 27th to Friday 29th October. It was a kind of unusual conference for me – No data focus, no majority of Azure data or Power BI sessions – it was a conference about Business Central (Dynamics). AND – I learned a lot. For me, now it is clearer what Business Central is capable of, how to get data into Business Central and how to get data out of BC to perform data analysis upon that data.
Right after the registration – Full mask mode during the conference
My Sessions: Power BI, Azure Synapse, PowerPlatform Dataflows
I had the chance to present two sessions about data anyway.
T-10 minutes before the first sessionT-15 min before the PowerPlatform dataflows session
The first session was about the relationship of Azure Synapse Analytics and Power BI. During this session I compared the two tools and how they can be used in the flow of data – from the datasource to data visualization.
Session 1 – Power BI and Azure Synapse – better together?
The second session was about PowerPlatform dataflows. I described the concept of dataflows, why they are needed and why they have been introduced. Next I compared the possibilities of Power BI dataflows versus PowerPlatform dataflows.
Session 2 – What are the possibilities of PowerPlatform dataflows?
What I really liked at the conference was the corona-save health procedures – temperature check plus check of the “green pass” everytime you entered the conference center. Plus mandatory wearing a mask anytime within a building..
Sie gehören zum Herbst dazu – die SQLDays der PPEDV in Erding. Traditionell in der Stadthalle in Erding und heuer auch endlich wieder in-person oder genauer gesagt als hybride Veranstaltung.
Ein Teil der Teilnehmer war direkt vor Ort dabei und der Rest konnte per Teams-Meeting bei den Sessions dabei sein.
Nach der 3G Kontrolle am Eingang stand dann dem Vortragsvergnügen (hoffe ich mal) nichts mehr im Wege.
Ich war heuer mit meinen aktuellen Lieblingsthemen unterwegs
workingondata 