How to process data in ssis
up vote
0
down vote
favorite
Need to convert data from the below format
From
1,Delivery Confirmation Email
5,11/09/2018 07:10
7,OOLU1939921
9,S00224239
To
Event: Delivery Confirmation Email;
Date: 11/09/2018 07:10;
Container: OOLU1939921;
Job: S00224239
or in a table format as
Event Date Container Job
Delivery Confirmation Email 11/09/2018 07:10 OOLU1939921 S00224239
Is it possible through pivot in ssis?
sql ssis pivot
add a comment |
up vote
0
down vote
favorite
Need to convert data from the below format
From
1,Delivery Confirmation Email
5,11/09/2018 07:10
7,OOLU1939921
9,S00224239
To
Event: Delivery Confirmation Email;
Date: 11/09/2018 07:10;
Container: OOLU1939921;
Job: S00224239
or in a table format as
Event Date Container Job
Delivery Confirmation Email 11/09/2018 07:10 OOLU1939921 S00224239
Is it possible through pivot in ssis?
sql ssis pivot
what is the source type for data? Flat file, DB-Table or something else?
– im_one
Nov 5 at 2:29
it's a text file which i processed from an html file
– Drishya
Nov 5 at 2:33
What are the columns and rows? What are the names of the columns?
– Gordon Linoff
Nov 5 at 2:51
@Gordon Linoff : column names are Event,date,Container,Job row: Delivery confirmation emal, 11/09/2018,ooLUI939921,S00224239
– Drishya
Nov 5 at 2:55
@Gordon Linoff thanks..
– Drishya
Nov 5 at 2:58
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
Need to convert data from the below format
From
1,Delivery Confirmation Email
5,11/09/2018 07:10
7,OOLU1939921
9,S00224239
To
Event: Delivery Confirmation Email;
Date: 11/09/2018 07:10;
Container: OOLU1939921;
Job: S00224239
or in a table format as
Event Date Container Job
Delivery Confirmation Email 11/09/2018 07:10 OOLU1939921 S00224239
Is it possible through pivot in ssis?
sql ssis pivot
Need to convert data from the below format
From
1,Delivery Confirmation Email
5,11/09/2018 07:10
7,OOLU1939921
9,S00224239
To
Event: Delivery Confirmation Email;
Date: 11/09/2018 07:10;
Container: OOLU1939921;
Job: S00224239
or in a table format as
Event Date Container Job
Delivery Confirmation Email 11/09/2018 07:10 OOLU1939921 S00224239
Is it possible through pivot in ssis?
sql ssis pivot
sql ssis pivot
edited Nov 5 at 2:50
Gordon Linoff
739k32285388
739k32285388
asked Nov 5 at 2:08
Drishya
217
217
what is the source type for data? Flat file, DB-Table or something else?
– im_one
Nov 5 at 2:29
it's a text file which i processed from an html file
– Drishya
Nov 5 at 2:33
What are the columns and rows? What are the names of the columns?
– Gordon Linoff
Nov 5 at 2:51
@Gordon Linoff : column names are Event,date,Container,Job row: Delivery confirmation emal, 11/09/2018,ooLUI939921,S00224239
– Drishya
Nov 5 at 2:55
@Gordon Linoff thanks..
– Drishya
Nov 5 at 2:58
add a comment |
what is the source type for data? Flat file, DB-Table or something else?
– im_one
Nov 5 at 2:29
it's a text file which i processed from an html file
– Drishya
Nov 5 at 2:33
What are the columns and rows? What are the names of the columns?
– Gordon Linoff
Nov 5 at 2:51
@Gordon Linoff : column names are Event,date,Container,Job row: Delivery confirmation emal, 11/09/2018,ooLUI939921,S00224239
– Drishya
Nov 5 at 2:55
@Gordon Linoff thanks..
– Drishya
Nov 5 at 2:58
what is the source type for data? Flat file, DB-Table or something else?
– im_one
Nov 5 at 2:29
what is the source type for data? Flat file, DB-Table or something else?
– im_one
Nov 5 at 2:29
it's a text file which i processed from an html file
– Drishya
Nov 5 at 2:33
it's a text file which i processed from an html file
– Drishya
Nov 5 at 2:33
What are the columns and rows? What are the names of the columns?
– Gordon Linoff
Nov 5 at 2:51
What are the columns and rows? What are the names of the columns?
– Gordon Linoff
Nov 5 at 2:51
@Gordon Linoff : column names are Event,date,Container,Job row: Delivery confirmation emal, 11/09/2018,ooLUI939921,S00224239
– Drishya
Nov 5 at 2:55
@Gordon Linoff : column names are Event,date,Container,Job row: Delivery confirmation emal, 11/09/2018,ooLUI939921,S00224239
– Drishya
Nov 5 at 2:55
@Gordon Linoff thanks..
– Drishya
Nov 5 at 2:58
@Gordon Linoff thanks..
– Drishya
Nov 5 at 2:58
add a comment |
3 Answers
3
active
oldest
votes
up vote
1
down vote
Here is a similar question with answers suggesting a script task or an ELT approach
SSIS transformation (almost like a pivot)
You can try and implement the script transformation if you wish. Personally I prefer an ELT approach
- Create a staging table with three columns:
.
CREATE TABLE DeliveryConfirmationStaging
(
ID INT NOT NULL,
StringValue VARCHAR(50) NOT NULL,
DTInserted DATETIME2(0) DEFAULT(GETDATE())
)
Load your text file into that table with a CSV source and an ADO SQL Server destination in a data flow.
Add a SQL script before your data flow with
TRUNCATE TABLE DeliveryConfirmationStaging;
in itLastly add this SQL script after your data flow which transforms it with SQL (borrowing @GordonLinoff answer)
.
insert into yourfinaltable (event, date,container,job)
select max(case when col1 = 1 then col2 end) as event,
TRY_CAST(max(case when col1 = 5 then col2 end) as datetime) as [date],
max(case when col1 = 7 then col2 end) as container,
max(case when col1 = 9 then col2 end) as job
from DeliveryConfirmationStaging;
You don't mention what date format this is. Is it September 11th or 9th November?
add a comment |
up vote
0
down vote
I think you can use conditional aggregation:
select max(case when col1 = 1 then col2 end) as event,
max(case when col1 = 5 then col2 end) as date,
max(case when col1 = 7 then col2 end) as container,
max(case when col1 = 9 then col2 end) as job
from t;
did you mean the above code can be implemented in ssis? Sorry I am newbie to SSIS and i couldn't find an option of conditional aggregation.
– Drishya
Nov 5 at 3:21
@Drishya . . . Your question is tagged "sql", so I assume a SQL solution meets your needs.
– Gordon Linoff
Nov 5 at 13:02
add a comment |
up vote
0
down vote
Fixed the problem through Pivot transformation in ssis.
add a comment |
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
1
down vote
Here is a similar question with answers suggesting a script task or an ELT approach
SSIS transformation (almost like a pivot)
You can try and implement the script transformation if you wish. Personally I prefer an ELT approach
- Create a staging table with three columns:
.
CREATE TABLE DeliveryConfirmationStaging
(
ID INT NOT NULL,
StringValue VARCHAR(50) NOT NULL,
DTInserted DATETIME2(0) DEFAULT(GETDATE())
)
Load your text file into that table with a CSV source and an ADO SQL Server destination in a data flow.
Add a SQL script before your data flow with
TRUNCATE TABLE DeliveryConfirmationStaging;
in itLastly add this SQL script after your data flow which transforms it with SQL (borrowing @GordonLinoff answer)
.
insert into yourfinaltable (event, date,container,job)
select max(case when col1 = 1 then col2 end) as event,
TRY_CAST(max(case when col1 = 5 then col2 end) as datetime) as [date],
max(case when col1 = 7 then col2 end) as container,
max(case when col1 = 9 then col2 end) as job
from DeliveryConfirmationStaging;
You don't mention what date format this is. Is it September 11th or 9th November?
add a comment |
up vote
1
down vote
Here is a similar question with answers suggesting a script task or an ELT approach
SSIS transformation (almost like a pivot)
You can try and implement the script transformation if you wish. Personally I prefer an ELT approach
- Create a staging table with three columns:
.
CREATE TABLE DeliveryConfirmationStaging
(
ID INT NOT NULL,
StringValue VARCHAR(50) NOT NULL,
DTInserted DATETIME2(0) DEFAULT(GETDATE())
)
Load your text file into that table with a CSV source and an ADO SQL Server destination in a data flow.
Add a SQL script before your data flow with
TRUNCATE TABLE DeliveryConfirmationStaging;
in itLastly add this SQL script after your data flow which transforms it with SQL (borrowing @GordonLinoff answer)
.
insert into yourfinaltable (event, date,container,job)
select max(case when col1 = 1 then col2 end) as event,
TRY_CAST(max(case when col1 = 5 then col2 end) as datetime) as [date],
max(case when col1 = 7 then col2 end) as container,
max(case when col1 = 9 then col2 end) as job
from DeliveryConfirmationStaging;
You don't mention what date format this is. Is it September 11th or 9th November?
add a comment |
up vote
1
down vote
up vote
1
down vote
Here is a similar question with answers suggesting a script task or an ELT approach
SSIS transformation (almost like a pivot)
You can try and implement the script transformation if you wish. Personally I prefer an ELT approach
- Create a staging table with three columns:
.
CREATE TABLE DeliveryConfirmationStaging
(
ID INT NOT NULL,
StringValue VARCHAR(50) NOT NULL,
DTInserted DATETIME2(0) DEFAULT(GETDATE())
)
Load your text file into that table with a CSV source and an ADO SQL Server destination in a data flow.
Add a SQL script before your data flow with
TRUNCATE TABLE DeliveryConfirmationStaging;
in itLastly add this SQL script after your data flow which transforms it with SQL (borrowing @GordonLinoff answer)
.
insert into yourfinaltable (event, date,container,job)
select max(case when col1 = 1 then col2 end) as event,
TRY_CAST(max(case when col1 = 5 then col2 end) as datetime) as [date],
max(case when col1 = 7 then col2 end) as container,
max(case when col1 = 9 then col2 end) as job
from DeliveryConfirmationStaging;
You don't mention what date format this is. Is it September 11th or 9th November?
Here is a similar question with answers suggesting a script task or an ELT approach
SSIS transformation (almost like a pivot)
You can try and implement the script transformation if you wish. Personally I prefer an ELT approach
- Create a staging table with three columns:
.
CREATE TABLE DeliveryConfirmationStaging
(
ID INT NOT NULL,
StringValue VARCHAR(50) NOT NULL,
DTInserted DATETIME2(0) DEFAULT(GETDATE())
)
Load your text file into that table with a CSV source and an ADO SQL Server destination in a data flow.
Add a SQL script before your data flow with
TRUNCATE TABLE DeliveryConfirmationStaging;
in itLastly add this SQL script after your data flow which transforms it with SQL (borrowing @GordonLinoff answer)
.
insert into yourfinaltable (event, date,container,job)
select max(case when col1 = 1 then col2 end) as event,
TRY_CAST(max(case when col1 = 5 then col2 end) as datetime) as [date],
max(case when col1 = 7 then col2 end) as container,
max(case when col1 = 9 then col2 end) as job
from DeliveryConfirmationStaging;
You don't mention what date format this is. Is it September 11th or 9th November?
edited Nov 5 at 9:33
answered Nov 5 at 6:33
Nick.McDermaid
11.5k32554
11.5k32554
add a comment |
add a comment |
up vote
0
down vote
I think you can use conditional aggregation:
select max(case when col1 = 1 then col2 end) as event,
max(case when col1 = 5 then col2 end) as date,
max(case when col1 = 7 then col2 end) as container,
max(case when col1 = 9 then col2 end) as job
from t;
did you mean the above code can be implemented in ssis? Sorry I am newbie to SSIS and i couldn't find an option of conditional aggregation.
– Drishya
Nov 5 at 3:21
@Drishya . . . Your question is tagged "sql", so I assume a SQL solution meets your needs.
– Gordon Linoff
Nov 5 at 13:02
add a comment |
up vote
0
down vote
I think you can use conditional aggregation:
select max(case when col1 = 1 then col2 end) as event,
max(case when col1 = 5 then col2 end) as date,
max(case when col1 = 7 then col2 end) as container,
max(case when col1 = 9 then col2 end) as job
from t;
did you mean the above code can be implemented in ssis? Sorry I am newbie to SSIS and i couldn't find an option of conditional aggregation.
– Drishya
Nov 5 at 3:21
@Drishya . . . Your question is tagged "sql", so I assume a SQL solution meets your needs.
– Gordon Linoff
Nov 5 at 13:02
add a comment |
up vote
0
down vote
up vote
0
down vote
I think you can use conditional aggregation:
select max(case when col1 = 1 then col2 end) as event,
max(case when col1 = 5 then col2 end) as date,
max(case when col1 = 7 then col2 end) as container,
max(case when col1 = 9 then col2 end) as job
from t;
I think you can use conditional aggregation:
select max(case when col1 = 1 then col2 end) as event,
max(case when col1 = 5 then col2 end) as date,
max(case when col1 = 7 then col2 end) as container,
max(case when col1 = 9 then col2 end) as job
from t;
answered Nov 5 at 2:52
Gordon Linoff
739k32285388
739k32285388
did you mean the above code can be implemented in ssis? Sorry I am newbie to SSIS and i couldn't find an option of conditional aggregation.
– Drishya
Nov 5 at 3:21
@Drishya . . . Your question is tagged "sql", so I assume a SQL solution meets your needs.
– Gordon Linoff
Nov 5 at 13:02
add a comment |
did you mean the above code can be implemented in ssis? Sorry I am newbie to SSIS and i couldn't find an option of conditional aggregation.
– Drishya
Nov 5 at 3:21
@Drishya . . . Your question is tagged "sql", so I assume a SQL solution meets your needs.
– Gordon Linoff
Nov 5 at 13:02
did you mean the above code can be implemented in ssis? Sorry I am newbie to SSIS and i couldn't find an option of conditional aggregation.
– Drishya
Nov 5 at 3:21
did you mean the above code can be implemented in ssis? Sorry I am newbie to SSIS and i couldn't find an option of conditional aggregation.
– Drishya
Nov 5 at 3:21
@Drishya . . . Your question is tagged "sql", so I assume a SQL solution meets your needs.
– Gordon Linoff
Nov 5 at 13:02
@Drishya . . . Your question is tagged "sql", so I assume a SQL solution meets your needs.
– Gordon Linoff
Nov 5 at 13:02
add a comment |
up vote
0
down vote
Fixed the problem through Pivot transformation in ssis.
add a comment |
up vote
0
down vote
Fixed the problem through Pivot transformation in ssis.
add a comment |
up vote
0
down vote
up vote
0
down vote
Fixed the problem through Pivot transformation in ssis.
Fixed the problem through Pivot transformation in ssis.
answered yesterday
Drishya
217
217
add a comment |
add a comment |
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53147428%2fhow-to-process-data-in-ssis%23new-answer', 'question_page');
}
);
Post as a guest
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
what is the source type for data? Flat file, DB-Table or something else?
– im_one
Nov 5 at 2:29
it's a text file which i processed from an html file
– Drishya
Nov 5 at 2:33
What are the columns and rows? What are the names of the columns?
– Gordon Linoff
Nov 5 at 2:51
@Gordon Linoff : column names are Event,date,Container,Job row: Delivery confirmation emal, 11/09/2018,ooLUI939921,S00224239
– Drishya
Nov 5 at 2:55
@Gordon Linoff thanks..
– Drishya
Nov 5 at 2:58