Pentaho Load Plain Text File w/ ASCII separator
up vote
0
down vote
favorite
I'm trying to use Spoon / Kettle to upload a plain text file that is separated by ASCII characters. I can see all the data when I preview the content of the file in Kettle, but no records load when I try to preview rows on the "Content" tab.
According to my research, Kettle should understand my field separator when typed as "$[value]" which in my case is "$[01]". Here's a description of the file structure:
Each file in the feed is in plain text format, separated into columns and rows. Each record has the same set of fields. The following are the delimiters for
each field and record:
Field Separator (FS): SOH (ASCII character 1)
Record Separator (RS) : STX (ASCII character 2) + “n”
Any record starting with a “#” and ending with the RS should be treated as a comment by the ingester and ignored. The data provider has also generated a column header line at the beginning of the file, listing field data types.
So my input parameters are:
Filetype: Fixed
Separator: $[01]
Enclosure:
Escape:
...
Format: DOS
Encoding: US-ASCII
Length: Characters
I'm unable to read any records, and I'm not sure if this is the correct approach. Would ingesting this data with java inside of kettle be a better method?
Any help with this would be much appreciated. Thanks!
ascii pentaho etl pentaho-spoon pentaho-data-integration
asked Nov 7 at 22:43
Grant Bender
11
add a comment |
up vote
0
down vote
favorite
I'm trying to use Spoon / Kettle to upload a plain text file that is separated by ASCII characters. I can see all the data when I preview the content of the file in Kettle, but no records load when I try to preview rows on the "Content" tab.
According to my research, Kettle should understand my field separator when typed as "$[value]" which in my case is "$[01]". Here's a description of the file structure:
Each file in the feed is in plain text format, separated into columns and rows. Each record has the same set of fields. The following are the delimiters for
each field and record:
Field Separator (FS): SOH (ASCII character 1)
Record Separator (RS) : STX (ASCII character 2) + “n”
Any record starting with a “#” and ending with the RS should be treated as a comment by the ingester and ignored. The data provider has also generated a column header line at the beginning of the file, listing field data types.
So my input parameters are:
Filetype: Fixed
Separator: $[01]
Enclosure:
Escape:
...
Format: DOS
Encoding: US-ASCII
Length: Characters
I'm unable to read any records, and I'm not sure if this is the correct approach. Would ingesting this data with java inside of kettle be a better method?
Any help with this would be much appreciated. Thanks!
ascii pentaho etl pentaho-spoon pentaho-data-integration
asked Nov 7 at 22:43
Grant Bender
11
1
Could you please provide sample Data for the same?
– KP M
Nov 8 at 10:18
Depending on your O.S, windows or linux, there are different control characters to be monitored. if you want your separator to be the literal $[01], what you need to type is $[01] ... being the escape character.
– Cristian Curti
Nov 13 at 19:40
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I'm trying to use Spoon / Kettle to upload a plain text file that is separated by ASCII characters. I can see all the data when I preview the content of the file in Kettle, but no records load when I try to preview rows on the "Content" tab.
According to my research, Kettle should understand my field separator when typed as "$[value]" which in my case is "$[01]". Here's a description of the file structure:
Each file in the feed is in plain text format, separated into columns and rows. Each record has the same set of fields. The following are the delimiters for
each field and record:
Field Separator (FS): SOH (ASCII character 1)
Record Separator (RS) : STX (ASCII character 2) + “n”
Any record starting with a “#” and ending with the RS should be treated as a comment by the ingester and ignored. The data provider has also generated a column header line at the beginning of the file, listing field data types.
So my input parameters are:
Filetype: Fixed
Separator: $[01]
Enclosure:
Escape:
...
Format: DOS
Encoding: US-ASCII
Length: Characters
I'm unable to read any records, and I'm not sure if this is the correct approach. Would ingesting this data with java inside of kettle be a better method?
Any help with this would be much appreciated. Thanks!
ascii pentaho etl pentaho-spoon pentaho-data-integration
asked Nov 7 at 22:43
Grant Bender
11
I'm trying to use Spoon / Kettle to upload a plain text file that is separated by ASCII characters. I can see all the data when I preview the content of the file in Kettle, but no records load when I try to preview rows on the "Content" tab.
According to my research, Kettle should understand my field separator when typed as "$[value]" which in my case is "$[01]". Here's a description of the file structure:
Each file in the feed is in plain text format, separated into columns and rows. Each record has the same set of fields. The following are the delimiters for
each field and record:
Field Separator (FS): SOH (ASCII character 1)
Record Separator (RS) : STX (ASCII character 2) + “n”
Any record starting with a “#” and ending with the RS should be treated as a comment by the ingester and ignored. The data provider has also generated a column header line at the beginning of the file, listing field data types.
So my input parameters are:
Filetype: Fixed
Separator: $[01]
Enclosure:
Escape:
...
Format: DOS
Encoding: US-ASCII
Length: Characters
I'm unable to read any records, and I'm not sure if this is the correct approach. Would ingesting this data with java inside of kettle be a better method?
Any help with this would be much appreciated. Thanks!
ascii pentaho etl pentaho-spoon pentaho-data-integration
ascii pentaho etl pentaho-spoon pentaho-data-integration
asked Nov 7 at 22:43
Grant Bender
11
asked Nov 7 at 22:43
Grant Bender
11
asked Nov 7 at 22:43
Grant Bender
11
asked Nov 7 at 22:43
Grant Bender
11
asked Nov 7 at 22:43
Grant Bender
11
11
1
Could you please provide sample Data for the same?
– KP M
Nov 8 at 10:18
Depending on your O.S, windows or linux, there are different control characters to be monitored. if you want your separator to be the literal $[01], what you need to type is $[01] ... being the escape character.
– Cristian Curti
Nov 13 at 19:40
add a comment |
1
Could you please provide sample Data for the same?
– KP M
Nov 8 at 10:18
Depending on your O.S, windows or linux, there are different control characters to be monitored. if you want your separator to be the literal $[01], what you need to type is $[01] ... being the escape character.
– Cristian Curti
Nov 13 at 19:40
1
1
Could you please provide sample Data for the same?
– KP M
Nov 8 at 10:18
Could you please provide sample Data for the same?
– KP M
Nov 8 at 10:18
Depending on your O.S, windows or linux, there are different control characters to be monitored. if you want your separator to be the literal $[01], what you need to type is $[01] ... being the escape character.
– Cristian Curti
Nov 13 at 19:40
Depending on your O.S, windows or linux, there are different control characters to be monitored. if you want your separator to be the literal $[01], what you need to type is $[01] ... being the escape character.
– Cristian Curti
Nov 13 at 19:40
add a comment |
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53199030%2fpentaho-load-plain-text-file-w-ascii-separator%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
Could you please provide sample Data for the same?
– KP M
Nov 8 at 10:18
Depending on your O.S, windows or linux, there are different control characters to be monitored. if you want your separator to be the literal $[01], what you need to type is $[01] ... being the escape character.
– Cristian Curti
Nov 13 at 19:40