Reading\Writing Different file format in HDFS by using pyspark

Issue – How to read\write different file format in HDFS by using pyspark File Format Action Procedure example without compression text File Read sc.textFile() orders = sc.textFile(“/user/BDD/navnit/data-master/retail_db/orders”) Write rdd.saveAsTextFile() orders.saveAsTextFile(“/user/BDD/navnit/saveTextFile/orders”) sequence File Read sc.sequenceFile(ordersSF = sc.sequenceFile(‘/user/BDD/navnit/saveSequenceFile/orders’) Write PipelinedRDD.saveAsSequenceFile() ordersKV.saveAsSequenceFile(‘/user/BDD/navnit/saveSequenceFile/orders’) Avro file Read sqlContext.read.format(“com.databricks.spark.avro”).load() orders = sqlContext.read.format(“com.databricks.spark.avro”).load(“/home/BDD/navnit/orders/”) Write dataFram.write.format(“com.databricks.spark.avro”).save() orders.write.format(“com.databricks.spark.avro”).save(“/user/BDD/navnit/saveAvroFile/orders”) Parquet File Read sqlContext.read.parquet() ordersParquet =… Read More Reading\Writing Different file format in HDFS by using pyspark

learnsqlteamLeave a comment

Cannot resolve the collation conflict between “xxx” and “xxx” in the equal to operation

Issue – Customer was getting below error when he was running any query. Msg 468, Level 16, State 9, Line 36 Cannot resolve the collation conflict between “SQL_Latin1_General_CP1_CI_AS” and “SQL_Latin1_General_CP1_CS_AS” in the equal to operation. Back ground – Table was part of replication and when he was running query on subscribe he was getting collation conflict… Read More Cannot resolve the collation conflict between “xxx” and “xxx” in the equal to operation

learnsqlteam1 Comment

Login failed for user ‘xxx’. Reason: Server is in single user mode. Only one administrator can connect at this time.

Issue – Login failed for user ‘xxx’. Reason: Server is in single user mode. Only one administrator can connect at this time. Cause – Ensure SQL Server is not running in Single user mode by checking start-up parameter. if you see -m; it mean your SQL is running with single user mode. -m : Starts an… Read More Login failed for user ‘xxx’. Reason: Server is in single user mode. Only one administrator can connect at this time.

learnsqlteamLeave a comment

The target database, ‘xxx’, is participating in an availability group and is currently not accessible for queries.

Issue – The target database, ‘xxx’, is participating in an availability group and is currently not accessible for queries. Either data movement is suspended or the availability replica is not enabled for read access. To allow read-only access to this and other databases in the availability group, enable read access to one or more secondary… Read More The target database, ‘xxx’, is participating in an availability group and is currently not accessible for queries.

learnsqlteamLeave a comment

Microsoft Contoso BI Demo Dataset for Retail Industry

Today, I was working on demo where I wanted to have test dataset for Retail db and after some googing I found MS has it’s own database and here are details – The Contoso BI Demo dataset is used to demonstrate DW/BI functionalities across the entire Microsoft Office product family. This dataset includes C-level, sales/marketing,… Read More Microsoft Contoso BI Demo Dataset for Retail Industry

learnsqlteamLeave a comment

The transaction log for database ‘XXX’ is full due to ‘LOG_BACKUP’

Yesterday, one of my client send me email stating she is not able to run any query against database. she was getting below error message – The transaction log for database ‘xxx’ is full due to ‘LOG_BACKUP’ From error message it’s look like transaction log full issue hence I looked for db property and found… Read More The transaction log for database ‘XXX’ is full due to ‘LOG_BACKUP’

learnsqlteam1 Comment

Performance Dash board and performance Baseline

These two articles were published my Microsoft SQL team and it will help you to create Performance dashboard and performance baseline report. These reports are really cool and will eliminate your performance monitoring need. https://blogs.msdn.microsoft.com/sql_server_team/sql-server-performance-dashboard-reports-unleashed-for-enterprise-monitoring/  https://blogs.msdn.microsoft.com/sql_server_team/sql-server-performance-baselining-reports-unleashed-for-enterprise-monitoring/#comment-2705

learnsqlteamLeave a comment

Server TCP provider failed to listen on [ ‘any’ 1433]. Tcp port is already in use.

Issue – Today I faced very weird issue; on multi instance SQL Server machine, couple of SQL Services was not starting after Server reboot ( In my case it was SQL 2014 and SQL 2016 ) and every time I was trying to restart my SQL Services, I was getting below error message –  … Read More Server TCP provider failed to listen on [ ‘any’ 1433]. Tcp port is already in use.

learnsqlteamLeave a comment

Cannot be configured for publishing and distribution. Publishing and distribution are supported only in SQL Server version 7.0 or later.

I got really wired issue when i was trying to configure Replication on my Enterprise edition 2016 server. I checked couple of time and everything was looking okay until I realize I was running SSMS 2014. I started SSMS 2016 and it’s fixed my issue. I know its not very big issue but I thought… Read More Cannot be configured for publishing and distribution. Publishing and distribution are supported only in SQL Server version 7.0 or later.

learnsqlteam1 Comment

The server principal “XYZ\USER” is not able to access the database “YourDBName” under the current security context

My Development team contacted me and said they are not able to access database  “The server principal “XYZ\USER” is not able to access the database “YourDBName” under the current security context” when I looked into server server I found – Login was there and access was granted. When I looked into database I found user… Read More The server principal “XYZ\USER” is not able to access the database “YourDBName” under the current security context

learnsqlteamLeave a comment