SQLPartitionMgmt Forum Rss Feed

New Post: Source Code

MikeHoney — Thu, 13 Jun 2013 05:03:30 GMT

Sorry I just found it on the Downloads page. I was looking under Source Code ...

New Post: Source Code

MikeHoney — Thu, 13 Jun 2013 04:59:13 GMT

The source code for this project does not appear to be published?

New Post: Can you add a "SplitPartition" function?

MikeHoney — Tue, 26 Mar 2013 22:45:12 GMT

In my current SSIS package l have an Execute SQL Task preceeding a call to ManagePartition.exe with the ClearPartition command. I've shown the code below.

There's some complexity there and it strikes me the ManagePartition.exe already has all the parameters in place to assist with this.

Perhaps it would be a new "SplitPartition" function?

-- SQL_Split_Partition
-- Parameter mapping to variable - same as /PartitionRangeValue
IF NOT EXISTS(

        SELECT prv.value, pf.name
        FROM sys.partition_functions AS pf
        JOIN sys.partition_range_values AS prv ON
              prv.function_id = pf.function_id
        WHERE
              pf.name = 'MyPartitionFunction'
              AND CAST(prv.value AS datetime2) = ?
        )

BEGIN

-- Prepare to Split a partition
ALTER  PARTITION SCHEME  [MyPartitionScheme]
NEXT USED [PRIMARY]

-- Split a partition 
ALTER PARTITION FUNCTION [MyPartitionFunction] ()
SPLIT RANGE ( ? )

END

New Post: Can you add Standard Output messages?

MikeHoney — Tue, 26 Mar 2013 22:37:44 GMT

I think it would be reassuring if the application could produce messages summarizing what it has done, following a successful execution. These would be written to Standard Output.

I'm integrating it with SSIS via the Execute Process Task and would like to catch those messages and write them to my logging.

Thanks!
Mike

New Post: Can you add a "SwitchPartition" function?

MikeHoney — Thu, 21 Mar 2013 00:23:33 GMT

I think it would be useful to add a "SwitchPartition" function to avoid having to find the partition number and then call ALTER TABLE ... SWITCH following a staging table load.

For my SSIS package I had to:

define a variable for Partition Number
populate with the results of a variable-driven query e.g. SELECT CAST ( $partition.InvestmentStructureExposures AS NVARCHAR(250) )
use the Partition Number to generate another variable-driven SQL for the ALTER TABLE ... SWITCH

It strikes me that all the required sub-functions, parameters etc are already there to allow adding this function into the code. This would mean one last call to your module could take care of that and save a bit of package complexity.

Thanks
Mike

New Post: Documentation is a bit thin

MikeHoney — Thu, 21 Mar 2013 00:12:34 GMT

Hi Stuart

Yes it has actually worked first time and cut the execution time for one tricky table in half.

This table is currently 60m rows, 13GB data, 10GB indexes (Page Compression), representing about 200 days of data. It receives a varying set of around 12 dates to refresh each day, and previously was deleting then inserting (via SSIS), then rebuilding indexes. A perfect scenario for this component!

A test query across a range of dates now runs 3 times faster - another win.

Thanks again
Mike

New Post: Documentation is a bit thin

Stuarto — Wed, 20 Mar 2013 16:05:04 GMT

Thanks for the feedback Mike. Will try to expand discussion as suggested. Glad it has been a timesaver…

- Stuart

From: MikeHoney [email removed]
Sent: Tuesday, March 19, 2013 11:35 PM
To: Stuart Ozer
Subject: Documentation is a bit thin [SQLPartitionMgmt:437284]

From: MikeHoney

Thanks for putting this together - it has saved a lot of time.

I did find the documentation a bit thin, in particular there isnt much info on each parameter or any explanantion of what the examples will do. Theres also no example of the IndexStaging command.

It would be great if that could be expanded.

Thanks
Mike

New Post: Documentation is a bit thin

MikeHoney — Wed, 20 Mar 2013 06:34:46 GMT

New Post: Problem with Identity Columns

jortacua — Wed, 06 Feb 2013 12:31:50 GMT

I was recently working with the command ManagePartition /C: CreateStagingFull on a table in my database and what I've noticed is that it doesn't creates the Identity field on staging table.

May be a bug with the cloning of the source table?

Here is my script in sql to create the source table and then the use of ManagePartition:

CREATE TABLE [Core]. [PassValueHistory] (
[IdPassValue] [int] IDENTITY (1,1) NOT NULL,
[PartitionDate] [date] NOT NULL,
[IdPass] [dbo]. [IDENTIFIER] NOT NULL,
[IdVariable] [dbo]. [IDENTIFIER] NOT NULL,
[ValueDateTime] [datetime] NOT NULL,
[Value] [sql_variant] NULL,
CONSTRAINT [PK_PassValueHistory] PRIMARY KEY CLUSTERED
(
[IdPassValue] ASC,
[PartitionDate] ASC
) WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
)

ALTER TABLE [Core]. [PassValueHistory] WITH NOCHECK ADD CONSTRAINT [FK_PassValueHistory_Pass] FOREIGN KEY ([IdPass], [PartitionDate])
REFERENCES [Core]. [PassHistory] ([IdPass], [PartitionDate])
GO

ALTER TABLE [Core]. [PassValueHistory] NOCHECK CONSTRAINT [FK_PassValueHistory_Pass]
GO

ALTER TABLE [Core]. [PassValueHistory] WITH CHECK ADD CONSTRAINT [FK_PassValueHistory_Variable] FOREIGN KEY ([IdVariable])
REFERENCES [Core]. [Variable] ([IdVariable])
GO

ALTER TABLE [Core]. [PassValueHistory] CHECK CONSTRAINT [FK_PassValueHistory_Variable]
GO

C: > ManagePartition / C: CreateStagingFull / d: DbDw / s: Core / t: PassValueHistory / p: 1 / A: staging_ValorTubo

Many Thanks, Jorge

New Post: ALTER TABLE SWITCH statement failed..

stuarto — Thu, 29 Nov 2012 04:48:22 GMT

Fixed in version 3.0

New Post: ALTER TABLE SWITCH statement failed. Check constraints or partition function of source table xxx allows values that are not allowed by check constraints or partition function on target table yyy.

stuarto — Thu, 29 Nov 2012 04:47:48 GMT

Fixed in version 3.0

New Post: Does not seem to support TEXT type columns

stuarto — Thu, 29 Nov 2012 04:47:25 GMT

Fixed in version 3.0

New Post: SQL 2008 Scheme instead of FileGroup

stuarto — Thu, 29 Nov 2012 04:47:05 GMT

Fixed in version 3.0.

New Post: SQL 2008 Scheme instead of FileGroup

kilvroch — Thu, 22 Nov 2012 08:50:12 GMT

Hi Stuarto

I am attaching my archive table description for you.

I think my confusion came from

a) Running your utility to create the staging table,

b) Noticing that the ItemId in the staging table was not set to identity

c) Using SSMS to script the staging table as ‘drop & create’

a. this is where I found the constraints to be confusing.

The way I got the staging table to work for me was to

a) define it with no index or constraints

b) load the data

c) add index and constraints the constraints being in the attached stored procedure

d) switch it into the partition

e) delete index and constraints from staging table in preparation for next load

f) rebuild statistics on archive table

Eileen Eby

Svc Info Developer IV, Technical Design Team

Hewlett-Packard Company

+44 1327 264 057 / Tel

+44 7768 700 755 / Mobile

eileen.eby@hp.com / Email

44 Farndon Rd

Woodford Halse, Daventry, NN11 3TT

Please consider the environment before printing this email.

From: stuarto [email removed]
Sent: 21 November 2012 23:08
To: Eby, Eileen
Subject: Re: SQL 2008 Scheme instead of FileGroup [SQLPartitionMgmt:210371]

From: stuarto

I expect the problem is a bug in my tool that affects operation when your table contains a long text or long binary column. I tested you example on a simple table and received no errors. Can you send me or post the DDL for the original table and I will verify that my upcoming fix will address this situation?

New Post: SQL 2008 Scheme instead of FileGroup

stuarto — Wed, 21 Nov 2012 23:07:49 GMT

New Post: Does not seem to support TEXT type columns

stuarto — Wed, 21 Nov 2012 22:56:41 GMT

This behavior is a bug in the current code and affects tables with all long types such as Text, Ntext, varchar(max), varbinary(max), etc. I will be releasing a new version shortly with this fixed.

New Post: ALTER TABLE SWITCH statement failed. Check constraints or partition function of source table xxx allows values that are not allowed by check constraints or partition function on target table yyy.

kilvroch — Sat, 17 Nov 2012 13:53:37 GMT

Thank you ….

Eileen Eby

Svc Info Developer IV, Technical Design Team

Hewlett-Packard Company

+44 1327 264 057 / Tel

+44 7768 700 755 / Mobile

eileen.eby@hp.com / Email

44 Farndon Rd

Woodford Halse, Daventry, NN11 3TT

Please consider the environment before printing this email.

From: stuartozer [email removed]
Sent: 17 November 2012 05:57
To: Eby, Eileen
Subject: Re: ALTER TABLE SWITCH statement failed. Check constraints or partition function of source table xxx ... [SQLPartitionMgmt:402894]

From: stuartozer

I confirmed that Partition Management tool will always create the constraint WITH CHECK, instead of using NOCHECK, and in your test case that will work. (Manually editing the DDL to add a WITH NOCHECK to the partition-specific staging table's check constraint prevents SQL Server from verifying that any rows in the staging table meet the constraint condition, and will then not allow a switch.)

New Post: ALTER TABLE SWITCH statement failed. Check constraints or partition function of source table xxx allows values that are not allowed by check constraints or partition function on target table yyy.

stuartozer — Sat, 17 Nov 2012 05:56:47 GMT

New Post: ALTER TABLE SWITCH statement failed. Check constraints or partition function of source table xxx allows values that are not allowed by check constraints or partition function on target table yyy.

Stuarto — Fri, 16 Nov 2012 22:57:27 GMT

The error is that WITH NOCHECK is used in the Check Constraint definition. The script below will work if you use WITH CHECK instead when you add the constraint on the staging table. I’ll have to investigate why the tool is generating a NOCHECK in your case… I can’t think of a situation where a WITH NOCHECK constraint should be applied!

- Stuart

From: kilvroch [email removed]
Sent: Monday, November 12, 2012 9:06 AM
To: Stuart Ozer
Subject: ALTER TABLE SWITCH statement failed. Check constraints or partition function of source table xxx ... [SQLPartitionMgmt:402894]

From: kilvroch

Can someone please help? I have googled and found that the constraint 'is not null' is required as well as the constraint on the column.

My partition function and scheme:
CREATE PARTITION FUNCTION [ItemsArchivePF1](int) AS RANGE right FOR VALUES (20111006)
GO
CREATE PARTITION SCHEME [ItemsArchivePS1]
AS PARTITION [ItemsArchivePF1] ALL TO ([PRIMARY])
GO

My partitioned table's DROP/DEFINE
SET ANSI_NULLS ON
GO

SET QUOTED_IDENTIFIER ON
GO

SET ANSI_PADDING ON
GO

CREATE TABLE [dbo].[tblItemsArchive](
[ItemDateId] [int] NOT NULL,
[ItemSourceBank] [char](3) NOT NULL,
[ItemFileId] [int] NOT NULL,
[ItemId] [bigint] identity(1,1) NOT NULL,
[ItemDocType] [char](1) NOT NULL,
[ItemIsn] [char](12) NOT NULL,
[ItemPkt] [char](2) NOT NULL,
[ItemRejectReason] [char](2) NOT NULL,
...
[ItemKB_Count] [int] NOT NULL
CONSTRAINT itemPK PRIMARY KEY CLUSTERED (ItemDateId, ItemSourceBank, ItemId)
) ON [ItemsArchivePS1]([ItemDateId])
WITH
(
DATA_COMPRESSION = PAGE
)

SET ANSI_PADDING OFF
GO

ObjectName IndexName IndexID PartitionScheme PartitionNumber FileGroupName LowerBoundaryValue UpperBoundaryValue Range Rows
tblItemsArchive itemPK 1 ItemsArchivePS1 1 PRIMARY NULL 20111006 RIGHT 0
tblItemsArchive itemPK 1 ItemsArchivePS1 2 PRIMARY 20111006 NULL RIGHT 0

When I run PartitionManager, the staging table created does not make the ItemId an identity field, so I manually changed it.
It is important for the ItemId to be an identity field in the daily table, but in the archive, it is okay not to be unique.
The constraint generated by PartitionManager reads:
ALTER TABLE [dbo].[staging_tblItemsArchive] WITH NOCHECK ADD CONSTRAINT [chk_myStagingTbl_partition_2] CHECK (([ItemDateId]>=(20111006)))

From googling other people who have had the same experience have added the equivalent of "[ItemDateId is not null and " to the constraint.
I have tried both ways and am going absolutley nuts.
What am I doing wrong?
My tables do not allow nulls in any of the fields.
Can my problem be with those 'ansi_padding' and 'ansi_nulls' statements?
Can my problem be with the identity column?

I use SSIS to load the staging table from a flat file, then attempt to switch it to partition 2.

I get:
ALTER TABLE staging_tblItemsArchive SWITCH TO tblItemsArchive PARTITION 2 ;
Msg 4972, Level 16, State 1, Line 1
ALTER TABLE SWITCH statement failed. Check constraints or partition function of source table 'PATS.dbo.staging_tblItemsArchive' allows values that are not allowed by check constraints or partition function on target table 'PATS.dbo.tblItemsArchive'.

I am pulling my hair out.

New Post: ALTER TABLE SWITCH statement failed. Check constraints or partition function of source table xxx allows values that are not allowed by check constraints or partition function on target table yyy.

kilvroch — Mon, 12 Nov 2012 17:05:28 GMT

Can someone please help? I have googled and found that the constraint 'is not null' is required as well as the constraint on the column.

My partitioned table's DROP/DEFINE
SET ANSI_NULLS ON
GO

SET QUOTED_IDENTIFIER ON
GO

SET ANSI_PADDING ON
GO

SET ANSI_PADDING OFF
GO

I use SSIS to load the staging table from a flat file, then attempt to switch it to partition 2.

I am pulling my hair out.