Issue – How to read\write different file format in HDFS by using pyspark File Format Action Procedure example without compression text File Read sc.textFile() orders = sc.textFile(“/user/BDD/navnit/data-master/retail_db/orders”) Write rdd.saveAsTextFile() orders.saveAsTextFile(“/user/BDD/navnit/saveTextFile/orders”) sequence File Read sc.sequenceFile(ordersSF = sc.sequenceFile(‘/user/BDD/navnit/saveSequenceFile/orders’) Write PipelinedRDD.saveAsSequenceFile() ordersKV.saveAsSequenceFile(‘/user/BDD/navnit/saveSequenceFile/orders’) Avro file Read sqlContext.read.format(“com.databricks.spark.avro”).load() orders = sqlContext.read.format(“com.databricks.spark.avro”).load(“/home/BDD/navnit/orders/”) Write dataFram.write.format(“com.databricks.spark.avro”).save() orders.write.format(“com.databricks.spark.avro”).save(“/user/BDD/navnit/saveAvroFile/orders”) Parquet File Read sqlContext.read.parquet() ordersParquet =… Read More Reading\Writing Different file format in HDFS by using pyspark
Issue – Customer was getting below error when he was running any query. Msg 468, Level 16, State 9, Line 36 Cannot resolve the collation conflict between “SQL_Latin1_General_CP1_CI_AS” and “SQL_Latin1_General_CP1_CS_AS” in the equal to operation. Back ground – Table was part of replication and when he was running query on subscribe he was getting collation conflict… Read More Cannot resolve the collation conflict between “xxx” and “xxx” in the equal to operation
Issue – How to export .dtsx file from SSISDB as from SQL 2012 onward you will not see .dtsx package listed in integration services. Note – This is shortcut solution. Step 1: – Go to Integration Services Catalogs à Project Step 2: – Right click on Project à Export Step 3: – Save project file… Read More How to export SSIS package ( dtsx file) from SSISDB
Requirement – If you are planning to migrate your In House SQL Server to Amazon Cloud and wondering which option will be best for you b/w AMAZON RDS and SQL Server on EC2. Here, I have created comparison table based on AWS whitepaper and documents. Feature comparison :- Features RDS SQL on EC2 Contol AWS In house… Read More Comparison between AMAZON RDS and SQL Server on EC2
Issue – Login failed for user ‘xxx’. Reason: Server is in single user mode. Only one administrator can connect at this time. Cause – Ensure SQL Server is not running in Single user mode by checking start-up parameter. if you see -m; it mean your SQL is running with single user mode. -m : Starts an… Read More Login failed for user ‘xxx’. Reason: Server is in single user mode. Only one administrator can connect at this time.
Issue – The target database, ‘xxx’, is participating in an availability group and is currently not accessible for queries. Either data movement is suspended or the availability replica is not enabled for read access. To allow read-only access to this and other databases in the availability group, enable read access to one or more secondary… Read More The target database, ‘xxx’, is participating in an availability group and is currently not accessible for queries.
Today, I was working on demo where I wanted to have test dataset for Retail db and after some googing I found MS has it’s own database and here are details – The Contoso BI Demo dataset is used to demonstrate DW/BI functionalities across the entire Microsoft Office product family. This dataset includes C-level, sales/marketing,… Read More Microsoft Contoso BI Demo Dataset for Retail Industry
Yesterday, one of my client send me email stating she is not able to run any query against database. she was getting below error message – The transaction log for database ‘xxx’ is full due to ‘LOG_BACKUP’ From error message it’s look like transaction log full issue hence I looked for db property and found… Read More The transaction log for database ‘XXX’ is full due to ‘LOG_BACKUP’
This is really awesome feature launched my Microsoft in SQL Server 2017 (vNext ) source – Microsoft Cloud Platform
These two articles were published my Microsoft SQL team and it will help you to create Performance dashboard and performance baseline report. These reports are really cool and will eliminate your performance monitoring need. https://blogs.msdn.microsoft.com/sql_server_team/sql-server-performance-dashboard-reports-unleashed-for-enterprise-monitoring/ https://blogs.msdn.microsoft.com/sql_server_team/sql-server-performance-baselining-reports-unleashed-for-enterprise-monitoring/#comment-2705
Issue – Today I faced very weird issue; on multi instance SQL Server machine, couple of SQL Services was not starting after Server reboot ( In my case it was SQL 2014 and SQL 2016 ) and every time I was trying to restart my SQL Services, I was getting below error message – … Read More Server TCP provider failed to listen on [ ‘any’ 1433]. Tcp port is already in use.
I got call from one of my client when I was about to leave office and they said their missing critical databases is not coming into multi_user mode. I was like it’s going to be very easy to resolve. I tried to run Alter database <dbname> with multi_user and got below error. Msg 1205, Level… Read More Alter database set multi_user not working
I got really wired issue when i was trying to configure Replication on my Enterprise edition 2016 server. I checked couple of time and everything was looking okay until I realize I was running SSMS 2014. I started SSMS 2016 and it’s fixed my issue. I know its not very big issue but I thought… Read More Cannot be configured for publishing and distribution. Publishing and distribution are supported only in SQL Server version 7.0 or later.
My Development team contacted me and said they are not able to access database “The server principal “XYZ\USER” is not able to access the database “YourDBName” under the current security context” when I looked into server server I found – Login was there and access was granted. When I looked into database I found user… Read More The server principal “XYZ\USER” is not able to access the database “YourDBName” under the current security context