Load Ordering for Dimension and Fact tables

Hi ,

I have situation where I get data from SRC Flat file and have to load Dimensional table and also fact table, using same data flow(have no other choice since I have to unpivot some src data). Since I have to load both tables in same data flow, I have to have a way to put load ordering constraint (I know informatica allows that). Does any one have any idea on how this can be done in SSIS?

I would be really grateful.

Thanks

The SSIS package designer contains a Control Flow tab and a Data Flow Tab. On the Control Flow tab, you would create two Data Flow tasks linked by a precedence constraint. The first task would load the dimension data and the second would load the fact data only if the first task succeeds or completes depending on the precedence conditions you configure.

Was your question this elementary?

|||In my question I said, I can't use two data flows, I have to use only one data flow. So is there a way to this?

Thanks,|||

DW Developer wrote:

Hi ,

I have situation where I get data from SRC Flat file and have to load Dimensional table and also fact table, using same data flow(have no other choice since I have to unpivot some src data). Since I have to load both tables in same data flow, I have to have a way to put load ordering constraint (I know informatica allows that). Does any one have any idea on how this can be done in SSIS?

I would be really grateful.

Thanks

Very very good question. The feature you are referring to is sometimes called "Intrinsic Flow Priority". It doesn't exist in SSIS at the moment and I hope to god they put it into the next release. I have requested it here: https://connect.microsoft.com/SQLServer/feedback/ViewFeedback.aspx?FeedbackID=178058 and have noted that it exists in Informatica. I would appreicate it if you could click-through and add some comments. We're more likely to get it if more people ask for it and give real reasons why they need it.

In the meantime, you can achieve the same using raw files to pass data between different data-flows. This is explained here:

Splitting order detail and order header information from one file into multiple tables
(http://blogs.conchango.com/jamiethomson/archive/2006/05/22/3974.aspx)

HTH

-Jamie

Load on SQL Server Database

Hi,
How do we find the load on the SQL server database. even i am not
asking in the right mannar but i will can explain my problem then u
suggest me the right way to clear my problem.
i have written my code in C++,i am using multi threading and my
database is SQL Server.
i closed the connections perfectly still, i am seeing the connection
leak on Performance Viewer.
So i disabled some of the SQL Statements so very very very very slowly
, the connection leaks are growing. like propostional to disabling the
SQL Statements growing of connection leak.
so i thought it will be load balancing factor of the SQL Server or my
thing will be worng please any body help me.
In short, I want to know if the load on the SQL Server can cause
connection leaks? and if yes, then how can we define load on the SQL
Server database.
Thanks in advance,
RSTake a look at Perfomnace Monitor
http://www.sql-server-performance.com/performance_monitor_counters_memory.asp
<sk.rasheedfarhan@.gmail.com> wrote in message
news:1167284187.612926.263370@.73g2000cwn.googlegroups.com...
> Hi,
> How do we find the load on the SQL server database. even i am not
> asking in the right mannar but i will can explain my problem then u
> suggest me the right way to clear my problem.
> i have written my code in C++,i am using multi threading and my
> database is SQL Server.
> i closed the connections perfectly still, i am seeing the connection
> leak on Performance Viewer.
> So i disabled some of the SQL Statements so very very very very slowly
> , the connection leaks are growing. like propostional to disabling the
> SQL Statements growing of connection leak.
> so i thought it will be load balancing factor of the SQL Server or my
> thing will be worng please any body help me.
> In short, I want to know if the load on the SQL Server can cause
> connection leaks? and if yes, then how can we define load on the SQL
> Server database.
> Thanks in advance,
> RS
>sql

Load Multiple Signed Assemblies

I am trying to load multiple strongly named assemblies into the same database which are signed with the same .snk file (signed in Visual Studio). I use the following code to create an asymmetric key and login as Books Online recommends:

CREATE ASYMMETRIC KEY SQLCLRKey FROM FILE = 'D:\dba\bin\Assembly.dll'

CREATE LOGIN CLRAssembler FROM ASYMMETRIC KEY SQLCLRKey

GRANT UNSAFE ASSEMBLY TO CLRAssembler

GRANT EXTERNAL ACCESS ASSEMBLY TO CLRAssembler

REVOKE CONNECT SQL FROM CLRAssembler

Do I need to create a new login and asymmetric key for each assembly I load? If so, do I need to sign each with a different key because its giving me an error message when I try to create 2 separate asymmetric keys/logins from 2 different assemblies which have been signed with the same .snk file.

The only way I've gotten everything to load properly is to create a separate key for each assembly and sign each, then create separate logins and asymmetric keys in the database.

Is this the only way to do this? Or am I missing something?

First of all I think you mean:

CREATE ASYMMETRIC KEY SQLCLRKey FROM EXECUTABLE FILE = 'D:\dba\bin\Assembly.dll'

FROM FILE = '...' requires a file that has both the public and private key in it, but an assembly has only the public key in it. Also you should be creating this key in the master database.

In order to use an asymmetic key to enable an assembly to be loaded the asymmetric key must be the master database and include public key, but the private key is not required. When FROM EXECUTABLE FILE = '...' is used the only the public key for the asymmetric key is saved. This key can be used to create a login to grant usafe assembly to. Then, assuming the use has the other appropriate permissions, any assembly signed with this key can be loaded with permission_set = unsafe. A single login is used to load all of the assemblies that are signed with the same key... you can't load the same asymmetric key more than once in the same database. You will have to be sure that Visual Studio is signing all your assemblies with the same key. If you are having to create a new login for each assembly it sounds like Visual Studio is creating a new key for each of these assemblies. When you go to the properties for your visual studio project browse for a common key, don't create a new one.

You can create the asymmetric key directly from the snk file that visual studio creates, for example if myKey.snk is the key pair created by visual studio then:

USE master
GO

CREATE ASYMMETRIC KEY [MyAssemblyKey] FROM FILE = 'c:\keys\myKey.snk'
-- remove the private key, no reason to leave it hanging around.
ALTER ASYMMETRIC KEY [MyAssemblyKey] REMOVE PRIVATE KEY

CREATE LOGIN [LoginMyAssemblyKey] FROM ASYMMETRIC KEY [Key MyAssemblyKey]
GRANT EXTERNAL ACCESS ASSEMBLY TO [LoginMyAssemblyKey]

GO

Once you have done this any assemblies signed with myKey.snk can be deployed from visual studio with unsafe permission set.

Dan

Dan

|||

My mistake. I did mean EXECUTABLE FILE.

I started out trying to sign them all with the same key and then loading them individually and dropping the key and login, however this was producing an error (which I can post once I get back into the office).

Do I need to load them in the same batch or script if I want to use the same login? Because I was running them separately.

If not, how do I specify the login to use? I tried using the AUTHORIZATION command with it and it threw a permissions error.

|||

I'm not sure what you mean when you say you drop the login after creating the assembly.

If you drop the login, or take away the login's USAFE ASSEMBLY permission, you will not be able to use the assembly even though even though it has been created. The login created with the assemblies key is required whenever any function from the assembly is used.

Dan

|||

You need only use CREATE ASSEMBLY. Authorization is used to specify an owner, it is not related to whether or not the assembly can be external acess or unsafe. If the assembly is being created WITH EXTERNAL_ACCESS or UNSAFE, SQL Server will use the key inside of the assembly to find the login created with that key, then check the permissions granted to that login. It, in effect, does this whenever a function from that assembly is used too.

Dan

Load images into Sql Server 2005

Hi There

I have not had much luck finding info in BOL for tsql or SSIS that tells one how to load images on the file system into sql server 2005.

All i have really been able to find is that IMAGE data type will not be used in future and that one should use a varbinary(max) data type.

I am thinking of using a for each file loop in SSIS, but then how do i load the images (.tif) into a sql server database table ? Perhaps i need to use a sql task with the filepath , or an active x script.

Anyway if anyone knows how i can load images from the file system into sql server 2005, please let me know.

Thanx

Hi,

U can use Activex Script. But it is better to use Script task.

|||

You should look at the Import Column component. http://msdn2.microsoft.com/en-us/library/ms141262.aspx

All you need is a source file or table, with a column containing the filenames you wish to insert. SSIS will then load each named file into the data flow, from whence you can insert it to the database as normal.

Choice of the best data type for storing images would be better discussed on the SQL Server engine forum.

Donald

|||

Hi Donald

Would a for each file loop not work better, as a source file or table is not realistic considering there are over 9 million flat files.

I am not sure i am following you, import column requires a data source, if i point a flat file source to a image file it gets very confused with randon columns and junk data. I am not sure how to feed the flat file image to the import column task?

I have figured out another way to do it. But i am also interested in your suggestion.

Thanx

|||

That is indeed a lot of files. Actually it is so many that you may want to contact me offline to discuss options. DonaldDotFarmerAtMicrosoftDotCom will get me.

One way to generate a suitable file as a source, may be to pipe a dir command to file and then parse the result using SSIS - either using a flat file source, or a script component. From there it would be relatively simple to implement your scenario.

Donald

|||

Thanx Donald

I am gonna try a couple of things and i will get back to you.

|||

The TextCopy utility is a great way to load binary data into BLOB fields. I am calling the util from a .bat file. Its another option anyway....

Check out this article

http://www.mssqlcity.com/Articles/KnowHow/Textcopy.htm

|||

Dietz wrote:

Hi There

I have not had much luck finding info in BOL for tsql or SSIS that tells one how to load images on the file system into sql server 2005.

All i have really been able to find is that IMAGE data type will not be used in future and that one should use a varbinary(max) data type.

I am thinking of using a for each file loop in SSIS, but then how do i load the images (.tif) into a sql server database table ? Perhaps i need to use a sql task with the filepath , or an active x script.

Anyway if anyone knows how i can load images from the file system into sql server 2005, please let me know.

Thanx

the t-sql OPENROWSET BULK statement is designed to import files into sql server: http://msdn2.microsoft.com/en-us/library/ms190312.aspx

this statement can then be used in an execute sql task.

i suggest that you consult with the t-sql forum if you need assistance with this statement.

Load images into Sql Server 2005

Hi There

I have not had much luck finding info in BOL for tsql or SSIS that tells one how to load images on the file system into sql server 2005.

All i have really been able to find is that IMAGE data type will not be used in future and that one should use a varbinary(max) data type.

I am thinking of using a for each file loop in SSIS, but then how do i load the images (.tif) into a sql server database table ? Perhaps i need to use a sql task with the filepath , or an active x script.

Anyway if anyone knows how i can load images from the file system into sql server 2005, please let me know.

Thanx

Hi,

U can use Activex Script. But it is better to use Script task.

|||

You should look at the Import Column component. http://msdn2.microsoft.com/en-us/library/ms141262.aspx

All you need is a source file or table, with a column containing the filenames you wish to insert. SSIS will then load each named file into the data flow, from whence you can insert it to the database as normal.

Choice of the best data type for storing images would be better discussed on the SQL Server engine forum.

Donald

|||

Hi Donald

Would a for each file loop not work better, as a source file or table is not realistic considering there are over 9 million flat files.

I am not sure i am following you, import column requires a data source, if i point a flat file source to a image file it gets very confused with randon columns and junk data. I am not sure how to feed the flat file image to the import column task?

I have figured out another way to do it. But i am also interested in your suggestion.

Thanx

|||

That is indeed a lot of files. Actually it is so many that you may want to contact me offline to discuss options. DonaldDotFarmerAtMicrosoftDotCom will get me.

One way to generate a suitable file as a source, may be to pipe a dir command to file and then parse the result using SSIS - either using a flat file source, or a script component. From there it would be relatively simple to implement your scenario.

Donald

|||

Thanx Donald

I am gonna try a couple of things and i will get back to you.

|||

The TextCopy utility is a great way to load binary data into BLOB fields. I am calling the util from a .bat file. Its another option anyway....

Check out this article

http://www.mssqlcity.com/Articles/KnowHow/Textcopy.htm

|||

Dietz wrote:

Hi There

I have not had much luck finding info in BOL for tsql or SSIS that tells one how to load images on the file system into sql server 2005.

All i have really been able to find is that IMAGE data type will not be used in future and that one should use a varbinary(max) data type.

I am thinking of using a for each file loop in SSIS, but then how do i load the images (.tif) into a sql server database table ? Perhaps i need to use a sql task with the filepath , or an active x script.

Anyway if anyone knows how i can load images from the file system into sql server 2005, please let me know.

Thanx

the t-sql OPENROWSET BULK statement is designed to import files into sql server: http://msdn2.microsoft.com/en-us/library/ms190312.aspx

this statement can then be used in an execute sql task.

i suggest that you consult with the t-sql forum if you need assistance with this statement.

Load image file

HI

how to load a image file in to a sql server.

do we need c# code for it or can we use just a t-sql procedure to upload it.

In my application we don't have ant c# or other application. we have only t-sql

thanks

sandipan

If you are planning to use SSIS, the Import Column transform could be useful for this purpose.

Thanks.

load from sqlserver to its instance

I need to load data from a table in sqlserver to a table
in an instance of the same server. The query need to be
run in the SQL server itself not in the instance. So...
insert into [servername\instancename].[dbname].
[dbo].table1 (
col1,
col2)
select
t2.col1,
t2.col2
from table2 as t2
I keep getting this message:
Server 'CNS-SFO-A46\QA' is not configured for DATA ACCESS.
Thanks in advance for you help.This is a multi-part message in MIME format.
--=_NextPart_000_038B_01C3D5F9.27781520
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
In Enterprise Mgr, right click on the linked server - CNS-SFO-A46\QA. Bring
up the properties, click on Server Options, then check the Data Access box.
--
Tom
---
Thomas A. Moreau, BSc, PhD, MCSE, MCDBA
SQL Server MVP
Columnist, SQL Server Professional
Toronto, ON Canada
www.pinnaclepublishing.com/sql
"Sandra" <anonymous@.discussions.microsoft.com> wrote in message
news:00db01c3d620$c2708f40$a601280a@.phx.gbl...
I need to load data from a table in sqlserver to a table
in an instance of the same server. The query need to be
run in the SQL server itself not in the instance. So...
insert into [servername\instancename].[dbname].
[dbo].table1 (
col1,
col2)
select
t2.col1,
t2.col2
from table2 as t2
I keep getting this message:
Server 'CNS-SFO-A46\QA' is not configured for DATA ACCESS.
Thanks in advance for you help.
--=_NextPart_000_038B_01C3D5F9.27781520
Content-Type: text/html;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
&

---

In Enterprise Mgr, right click on the =linked server - CNS-SFO-A46\QA. Bring up the properties, =click on Server Options, then check the Data Access box.
-- Tom
---T=homas A. Moreau, BSc, PhD, MCSE, MCDBASQL Server MVPColumnist, SQL =Server ProfessionalToronto, ON Canadahttp://www.pinnaclepublishing.com/sql">www.pinnaclepublishing.com=/sql
"Sandra" wrote in message news:00db01c3d620$c2=708f40$a601280a@.phx.gbl...I need to load data from a table in sqlserver to a table in an =instance of the same server. The query need to be run in the SQL server itself not =in the instance. So...insert into [servername\instancename].[dbname].[dbo].table1 (col1,col2)selectt2.col1,t2.col2from table2 as =t2 I keep getting this message:Server 'CNS-SFO-A46\QA' is =not configured for DATA ACCESS.Thanks in advance for you =help.

---

--=_NextPart_000_038B_01C3D5F9.27781520--|||I'm affraid I don't see a check box for Data Access. Is it
possible I'm looking in the wrong place?
>--Original Message--
>In Enterprise Mgr, right click on the linked server - CNS-
SFO-A46\QA. Bring
>up the properties, click on Server Options, then check
the Data Access box.
>--
>Tom
>----
--
>Thomas A. Moreau, BSc, PhD, MCSE, MCDBA
>SQL Server MVP
>Columnist, SQL Server Professional
>Toronto, ON Canada
>www.pinnaclepublishing.com/sql
>
>"Sandra" <anonymous@.discussions.microsoft.com> wrote in
message
>news:00db01c3d620$c2708f40$a601280a@.phx.gbl...
>I need to load data from a table in sqlserver to a table
>in an instance of the same server. The query need to be
>run in the SQL server itself not in the instance. So...
>insert into [servername\instancename].[dbname].
>[dbo].table1 (
>col1,
>col2)
>select
>t2.col1,
>t2.col2
>from table2 as t2
>I keep getting this message:
>Server 'CNS-SFO-A46\QA' is not configured for DATA ACCESS.
>Thanks in advance for you help.
>|||This is a multi-part message in MIME format.
--=_NextPart_000_0038_01C3D60C.8C53A730
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
Did you find the linked server under Security->Linked Servers?
--
Tom
----
Thomas A. Moreau, BSc, PhD, MCSE, MCDBA
SQL Server MVP
Columnist, SQL Server Professional
Toronto, ON Canada
www.pinnaclepublishing.com/sql
.
"Sandra" <anonymous@.discussions.microsoft.com> wrote in message
news:058101c3d631$7e232580$a101280a@.phx.gbl...
I'm affraid I don't see a check box for Data Access. Is it
possible I'm looking in the wrong place?
>--Original Message--
>In Enterprise Mgr, right click on the linked server - CNS-
SFO-A46\QA. Bring
>up the properties, click on Server Options, then check
the Data Access box.
>--
>Tom
>----
--
>Thomas A. Moreau, BSc, PhD, MCSE, MCDBA
>SQL Server MVP
>Columnist, SQL Server Professional
>Toronto, ON Canada
>www.pinnaclepublishing.com/sql
>
>"Sandra" <anonymous@.discussions.microsoft.com> wrote in
message
>news:00db01c3d620$c2708f40$a601280a@.phx.gbl...
>I need to load data from a table in sqlserver to a table
>in an instance of the same server. The query need to be
>run in the SQL server itself not in the instance. So...
>insert into [servername\instancename].[dbname].
>[dbo].table1 (
>col1,
>col2)
>select
>t2.col1,
>t2.col2
>from table2 as t2
>I keep getting this message:
>Server 'CNS-SFO-A46\QA' is not configured for DATA ACCESS.
>Thanks in advance for you help.
>
--=_NextPart_000_0038_01C3D60C.8C53A730
Content-Type: text/html;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
&

---

Did you find the linked server under Security->Linked Servers?
-- Tom
----Thomas A. =Moreau, BSc, PhD, MCSE, MCDBASQL Server MVPColumnist, SQL Server ProfessionalToronto, ON Canadahttp://www.pinnaclepublishing.com/sql">www.pinnaclepublishing.com=/sql.
"Sandra" wrote in message news:058101c3d631$7e=232580$a101280a@.phx.gbl...I'm affraid I don't see a check box for Data Access. Is it possible I'm =looking in the wrong place?>--Original Message-->In =Enterprise Mgr, right click on the linked server - CNS-SFO-A46\QA. Bring>up the properties, click on Server Options, then check =the Data Access box.>>-->Tom>>--=---->Thomas A. Moreau, BSc, PhD, MCSE, MCDBA>SQL Server MVP>Columnist, =SQL Server Professional>Toronto, ON Canada>www.pinnaclepublishing.com/sql>>>"Sand=ra" wrote in message>news:00db01c3d620$c2708f40$a601280a@.phx.gbl...>=I need to load data from a table in sqlserver to a table>in an instance =of the same server. The query need to be>run in the SQL server itself =not in the instance. So...>>insert into [servername\instancename].[dbname].>[dbo].table1 (>col1,>col2)>select>t2.col1,>t2.col2>from table2 as t2>>I keep getting this =message:>>Server 'CNS-SFO-A46\QA' is not configured for DATA =ACCESS.>>Thanks in advance for you help.>

---

--=_NextPart_000_0038_01C3D60C.8C53A730--|||Yes, I did and I don't find the name of the server listed
but when I try to create New Linked Server it says the
server already exist. The same response I get when I try
to run EXEC sp_addlinkedserver
@.server = 'servername\instancename'
Also, I see the name of the server in the sysservers table.
>--Original Message--
>Did you find the linked server under Security->Linked
Servers?
>--
> Tom
>----
>Thomas A. Moreau, BSc, PhD, MCSE, MCDBA
>SQL Server MVP
>Columnist, SQL Server Professional
>Toronto, ON Canada
>www.pinnaclepublishing.com/sql
>..
>"Sandra" <anonymous@.discussions.microsoft.com> wrote in
message
>news:058101c3d631$7e232580$a101280a@.phx.gbl...
>I'm affraid I don't see a check box for Data Access. Is it
>possible I'm looking in the wrong place?
>>--Original Message--
>>In Enterprise Mgr, right click on the linked server -
CNS-
>SFO-A46\QA. Bring
>>up the properties, click on Server Options, then check
>the Data Access box.
>>--
>>Tom
>>----
-
>--
>>Thomas A. Moreau, BSc, PhD, MCSE, MCDBA
>>SQL Server MVP
>>Columnist, SQL Server Professional
>>Toronto, ON Canada
>>www.pinnaclepublishing.com/sql
>>
>>"Sandra" <anonymous@.discussions.microsoft.com> wrote in
>message
>>news:00db01c3d620$c2708f40$a601280a@.phx.gbl...
>>I need to load data from a table in sqlserver to a table
>>in an instance of the same server. The query need to be
>>run in the SQL server itself not in the instance. So...
>>insert into [servername\instancename].[dbname].
>>[dbo].table1 (
>>col1,
>>col2)
>>select
>>t2.col1,
>>t2.col2
>>from table2 as t2
>>I keep getting this message:
>>Server 'CNS-SFO-A46\QA' is not configured for DATA
ACCESS.
>>Thanks in advance for you help.
>|||The server might already be listed under "remote servers". This would happen
for example if you were doing replication from one instance to another. You
can allow data access by running the procedure
sp_serveroption 'server' ,
'data access',
'true'
Hope this helps,
Joe Lax
<anonymous@.discussions.microsoft.com> wrote in message
news:06be01c3d641$bbad9740$a101280a@.phx.gbl...
> Yes, I did and I don't find the name of the server listed
> but when I try to create New Linked Server it says the
> server already exist. The same response I get when I try
> to run EXEC sp_addlinkedserver
> @.server = 'servername\instancename'
> Also, I see the name of the server in the sysservers table.
>
>
> >--Original Message--
> >Did you find the linked server under Security->Linked
> Servers?
> >
> >--
> > Tom
> >
> >----
> >Thomas A. Moreau, BSc, PhD, MCSE, MCDBA
> >SQL Server MVP
> >Columnist, SQL Server Professional
> >Toronto, ON Canada
> >www.pinnaclepublishing.com/sql
> >..
> >"Sandra" <anonymous@.discussions.microsoft.com> wrote in
> message
> >news:058101c3d631$7e232580$a101280a@.phx.gbl...
> >I'm affraid I don't see a check box for Data Access. Is it
> >possible I'm looking in the wrong place?
> >
> >>--Original Message--
> >>In Enterprise Mgr, right click on the linked server -
> CNS-
> >SFO-A46\QA. Bring
> >>up the properties, click on Server Options, then check
> >the Data Access box.
> >>
> >>--
> >>Tom
> >>
> >>----
> -
> >--
> >>Thomas A. Moreau, BSc, PhD, MCSE, MCDBA
> >>SQL Server MVP
> >>Columnist, SQL Server Professional
> >>Toronto, ON Canada
> >>www.pinnaclepublishing.com/sql
> >>
> >>
> >>"Sandra" <anonymous@.discussions.microsoft.com> wrote in
> >message
> >>news:00db01c3d620$c2708f40$a601280a@.phx.gbl...
> >>I need to load data from a table in sqlserver to a table
> >>in an instance of the same server. The query need to be
> >>run in the SQL server itself not in the instance. So...
> >>
> >>insert into [servername\instancename].[dbname].
> >>[dbo].table1 (
> >>col1,
> >>col2)
> >>select
> >>t2.col1,
> >>t2.col2
> >>from table2 as t2
> >>
> >>I keep getting this message:
> >>
> >>Server 'CNS-SFO-A46\QA' is not configured for DATA
> ACCESS.
> >>
> >>Thanks in advance for you help.
> >>
> >sql