SQL Server Partition Management

New Post: Source Code

MikeHoney — Thu, 13 Jun 2013 05:03:30 GMT

Sorry I just found it on the Downloads page. I was looking under Source Code ...

New Post: Source Code

MikeHoney — Thu, 13 Jun 2013 04:59:13 GMT

The source code for this project does not appear to be published?

New Post: Can you add a "SplitPartition" function?

MikeHoney — Tue, 26 Mar 2013 22:45:12 GMT

In my current SSIS package l have an Execute SQL Task preceeding a call to ManagePartition.exe with the ClearPartition command. I've shown the code below.

There's some complexity there and it strikes me the ManagePartition.exe already has all the parameters in place to assist with this.

Perhaps it would be a new "SplitPartition" function?

-- SQL_Split_Partition
-- Parameter mapping to variable - same as /PartitionRangeValue
IF NOT EXISTS(

        SELECT prv.value, pf.name
        FROM sys.partition_functions AS pf
        JOIN sys.partition_range_values AS prv ON
              prv.function_id = pf.function_id
        WHERE
              pf.name = 'MyPartitionFunction'
              AND CAST(prv.value AS datetime2) = ?
        )

BEGIN

-- Prepare to Split a partition
ALTER  PARTITION SCHEME  [MyPartitionScheme]
NEXT USED [PRIMARY]

-- Split a partition 
ALTER PARTITION FUNCTION [MyPartitionFunction] ()
SPLIT RANGE ( ? )

END

New Post: Can you add Standard Output messages?

MikeHoney — Tue, 26 Mar 2013 22:37:44 GMT

I think it would be reassuring if the application could produce messages summarizing what it has done, following a successful execution. These would be written to Standard Output.

I'm integrating it with SSIS via the Execute Process Task and would like to catch those messages and write them to my logging.

Thanks!
Mike

New Post: Can you add a "SwitchPartition" function?

MikeHoney — Thu, 21 Mar 2013 00:23:33 GMT

I think it would be useful to add a "SwitchPartition" function to avoid having to find the partition number and then call ALTER TABLE ... SWITCH following a staging table load.

For my SSIS package I had to:

define a variable for Partition Number
populate with the results of a variable-driven query e.g. SELECT CAST ( $partition.InvestmentStructureExposures AS NVARCHAR(250) )
use the Partition Number to generate another variable-driven SQL for the ALTER TABLE ... SWITCH

It strikes me that all the required sub-functions, parameters etc are already there to allow adding this function into the code. This would mean one last call to your module could take care of that and save a bit of package complexity.

Thanks
Mike

Reviewed: Partition Management Release 3.0 (Mar 20, 2013)

MikeHoney — Thu, 21 Mar 2013 00:14:13 GMT

Rated 5 Stars (out of 5) - Worked first time and cut load+index time for a tricky table in half.

New Post: Documentation is a bit thin

MikeHoney — Thu, 21 Mar 2013 00:12:34 GMT

Hi Stuart

Yes it has actually worked first time and cut the execution time for one tricky table in half.

This table is currently 60m rows, 13GB data, 10GB indexes (Page Compression), representing about 200 days of data. It receives a varying set of around 12 dates to refresh each day, and previously was deleting then inserting (via SSIS), then rebuilding indexes. A perfect scenario for this component!

A test query across a range of dates now runs 3 times faster - another win.

Thanks again
Mike

New Post: Documentation is a bit thin

Stuarto — Wed, 20 Mar 2013 16:05:04 GMT

Thanks for the feedback Mike. Will try to expand discussion as suggested. Glad it has been a timesaver…

- Stuart

From: MikeHoney [email removed]
Sent: Tuesday, March 19, 2013 11:35 PM
To: Stuart Ozer
Subject: Documentation is a bit thin [SQLPartitionMgmt:437284]

From: MikeHoney

Thanks for putting this together - it has saved a lot of time.

I did find the documentation a bit thin, in particular there isnt much info on each parameter or any explanantion of what the examples will do. Theres also no example of the IndexStaging command.

It would be great if that could be expanded.

Thanks
Mike

New Post: Documentation is a bit thin

MikeHoney — Wed, 20 Mar 2013 06:34:46 GMT

New Post: Problem with Identity Columns

jortacua — Wed, 06 Feb 2013 12:31:50 GMT

I was recently working with the command ManagePartition /C: CreateStagingFull on a table in my database and what I've noticed is that it doesn't creates the Identity field on staging table.

May be a bug with the cloning of the source table?

Here is my script in sql to create the source table and then the use of ManagePartition:

CREATE TABLE [Core]. [PassValueHistory] (
[IdPassValue] [int] IDENTITY (1,1) NOT NULL,
[PartitionDate] [date] NOT NULL,
[IdPass] [dbo]. [IDENTIFIER] NOT NULL,
[IdVariable] [dbo]. [IDENTIFIER] NOT NULL,
[ValueDateTime] [datetime] NOT NULL,
[Value] [sql_variant] NULL,
CONSTRAINT [PK_PassValueHistory] PRIMARY KEY CLUSTERED
(
[IdPassValue] ASC,
[PartitionDate] ASC
) WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
)

ALTER TABLE [Core]. [PassValueHistory] WITH NOCHECK ADD CONSTRAINT [FK_PassValueHistory_Pass] FOREIGN KEY ([IdPass], [PartitionDate])
REFERENCES [Core]. [PassHistory] ([IdPass], [PartitionDate])
GO

ALTER TABLE [Core]. [PassValueHistory] NOCHECK CONSTRAINT [FK_PassValueHistory_Pass]
GO

ALTER TABLE [Core]. [PassValueHistory] WITH CHECK ADD CONSTRAINT [FK_PassValueHistory_Variable] FOREIGN KEY ([IdVariable])
REFERENCES [Core]. [Variable] ([IdVariable])
GO

ALTER TABLE [Core]. [PassValueHistory] CHECK CONSTRAINT [FK_PassValueHistory_Variable]
GO

C: > ManagePartition / C: CreateStagingFull / d: DbDw / s: Core / t: PassValueHistory / p: 1 / A: staging_ValorTubo

Many Thanks, Jorge

Created Issue: bug in exception management [9870]

smurru — Thu, 24 Jan 2013 10:12:27 GMT

A new bug was introduced in ManagePartition.exe version 3.0.

The bug is located in the Run() method exceptionn handler at line number 235

```
}
catch (Exception e)
{
sc.Disconnect();
sc = null;
Console.WriteLine(e);
}
sc.Disconnect();
sc = null;

```
when an exception occurs, the exception handler sets sc = null
then the sc.Disconnect(); following the catch block fails with an "Object reference not set to an instance of an object", that is caught in the Main() exception handler level, instead of the real exception.

in version 2.0 a there was a throw at the end of the catch block.

```
catch (Exception ex)
{
sc.Disconnect();
sc = null;
throw ex;
}
sc.Disconnect();
sc = null;

```

Updated Release: Partition Management Release 3.0 (Nov 28, 2012)

stuarto — Thu, 29 Nov 2012 17:41:24 GMT

Release 3.0 adds support for SQL Server 2012 and is backward compatible with SQL Server 2008 and 2005. The release consists of:
• A Readme file
• The Executable
• The source code (Visual Studio project)

Enhancements include:
-- Support for Columnstore indexes in SQL Server 2012
-- Ability to create TSQL scripts for staging table and index creation operations
-- Full support for global date and time formats, locale independent
-- Support for binary partitioning column types
-- Fixes to issues reported in v2

Test scripts for SQL Server 2012, 2008 and 2005 are included.

Read the ReadMe file for more details, prerequisites, and documentation.

Released: Partition Management Release 3.0 (Nov 28, 2012)

Thu, 29 Nov 2012 17:41:24 GMT

New Post: ALTER TABLE SWITCH statement failed..

stuarto — Thu, 29 Nov 2012 04:48:22 GMT

Fixed in version 3.0

New Post: ALTER TABLE SWITCH statement failed. Check constraints or partition function of source table xxx allows values that are not allowed by check constraints or partition function on target table yyy.

stuarto — Thu, 29 Nov 2012 04:47:48 GMT

Fixed in version 3.0

New Post: Does not seem to support TEXT type columns

stuarto — Thu, 29 Nov 2012 04:47:25 GMT

Fixed in version 3.0

New Post: SQL 2008 Scheme instead of FileGroup

stuarto — Thu, 29 Nov 2012 04:47:05 GMT

Fixed in version 3.0.

Closed Issue: Invalid Constraints when server not using US format dates [5782]

Stuarto — Thu, 29 Nov 2012 04:46:22 GMT

Hello,
 
Love this util. But I had a major issue with it. Being outside the US of A, I don't use MM/dd/YYYY format dates. Because I run into a lot of US dates format/converting problems I set all my SQL servers to use ISO like date format, yyyy-MM-dd. Also avoids user confusion with reports..
 
I found that your util can create invalid constraints on the slide out tables (and presumable will do the same on the staging) as it inserts the constraints in American style formatted dates. This causes the util to create a slide out table that can't have partition data switched to. I've updated the CreateStgPartitionCheck() in the source code so that if the partition column is SqlDateType.DateTime (The only sql data type I use for partitioning..) then it will properly format the string and works ok for me now.
 
So now the construct for the min value predicate looks a little like this: 
 
// Construct the minimum value predicate string in the check constraint definition
 if (partitionNumber > 1)
 // 2010-01-22 Added By Yussuf Khan so it can cope with non US format DateTime formats (I don't use other datatypes for my partition columns so only coding datetime)
 if (partitionColumnType == SqlDataType.DateTime)
 {
 //now it seems for datetime it will return this in US format, dd/MM/yyyy
 //need to convert to more universal standard, yyyy-MM-dd , which SQL server detects ok no matter what the locale (well in my experience anyways!)
 dtDateBuilder = (DateTime)pf.RangeValues[partitionNumber - 2];
 strDateBuilder = dtDateBuilder.ToString("yyyy-MM-dd hh:mm:ss.fff");
 leftBoundary = partitionColumnName + ((pf.RangeType == RangeType.Right) ? ">=" : ">") + "N'" + strDateBuilder + "'";
 }
 else if (
 (partitionColumnType == SqlDataType.Date) ||
 (partitionColumnType == SqlDataType.Time) ||
 
I've done a similiar thing to the Max value string constraint builder too. I'm not really a .Net developer, and I've only tested a couple of times. But this does seem to fix. Any chance you can incorporate something like this fully?

Comments:

Fixed in Version 3.0

Updated Release: Partition Management Release 3.0 (Nov 28, 2012)

stuarto — Thu, 29 Nov 2012 04:44:19 GMT

Released: Partition Management Release 3.0 (Nov 28, 2012)

Thu, 29 Nov 2012 04:44:19 GMT