Data processing error solved by a copy paste?
1
I am encountering a very strange problem while processing data under Linux 16.04 using python 2.7.
I create a .csv file using this function :
from ast import literal_eval
with open('logs.csv') as f:
data = [literal_eval(line) for line in f]
the file is properly created and looks like that:
('2017-04-01 12:05:00','0.01770001','0.0177887','0.01780275','0.01770001')
('2017-04-01 12:10:00','0.0177887','0.01771308','0.01785263','0.01771039')
('2017-04-01 12:15:00','0.01773','0.01780092','0.01780092','0.01773')
('2017-04-01 12:20:00','0.0178','0.01781212','0.01784922','0.01774015')
('2017-04-01 12:25:00','0.01781212','0.01774528','0.01782994','0.01774528')
('2017-04-01 12:30:00','0.01774529','0.0178732','0.01788145','0.01774509')
('2017-04-01 12:35:00','0.01788145','0.01793318','0.01793318','0.01788145')
('2017-04-01 12:40:00','0.01794','0.01780093','0.01799984','0.01780092')
('2017-04-01 12:45:00','0.01785694','0.01806699','0.01807519','0.01785694')
('2017-04-01 12:50:00','0.01807999','0.01819687','0.01827573','0.018027')
('2017-04-01 12:55:00','0.01819687','0.01825402','0.0184','0.01800011')
('2017-04-01 13:00:00','0.01822416','0.01830994','0.01835554','0.0181777')
('2017-04-01 13:05:00','0.01825415','0.01810171','0.01830986','0.01810008')
('2017-04-01 13:10:00','0.01810174','0.01818991','0.01818991','0.01810173')
('2017-04-01 13:15:00','0.01818991','0.01818002','0.01819687','0.01818001')
('2017-04-01 13:20:00','0.01818002','0.01821999','0.01822','0.01818001')
I then pass it throught this code to draw a graph :
import numpy as np
import matplotlib.pyplot as plt
from matplotlib import dates, ticker
import matplotlib as mpl
from mpl_finance import candlestick_ohlc
from ast import literal_eval
mpl.style.use('default')
data =
ohlc_data =
with open('logsXMR.csv') as f:
data = [literal_eval(line) for line in f]
for line in data:
#ohlc_data.append((np.float64(line[0]), np.float64(line[1]), np.float64(line[2]), np.float64(line[3]), np.float64(line[4])))
ohlc_data.append((dates.datestr2num(line[0]), np.float64(line[1]), np.float64(line[2]), np.float64(line[3]), np.float64(line[4])))
fig, ax1 = plt.subplots()
candlestick_ohlc(ax1, ohlc_data, width = 0.5/((24*60)/5), colorup = 'g', colordown = 'r', alpha = 0.8)
#ax1.xaxis.set_major_formatter(dates.DateFormatter('%d/%m/%Y %H:%M'))
ax1.xaxis.set_major_locator(ticker.MaxNLocator(10))
plt.xticks(rotation = 30)
plt.grid()
plt.xlabel('Date')
plt.ylabel('Price')
plt.title('Historical Data XMRUSD')
plt.tight_layout()
plt.show()
But each time I get that error:
Traceback (most recent call last):
File "CSVing.py", line 15, in <module>
data = [literal_eval(line) for line in f]
File "/usr/lib/python2.7/ast.py", line 49, in literal_eval
node_or_string = parse(node_or_string, mode='eval')
File "/usr/lib/python2.7/ast.py", line 37, in parse
return compile(source, filename, mode, PyCF_ONLY_AST)
File "<unknown>", line 2
('2017-04-01 12:10:00','0.0177887','0.01771308','0.01785263','0.01771039')
^
I don't understand why I get this error because if I simply copy and paste my datas into another file, everything works fine, I can draw my graph flawlessly. I just don't get it because the 2 data files are identicals, no added space or anything.
what can cause this error and how can I proceed to be able to used my generated data file directly without the need to copy paste the datas in another file ?
thanks in advance,
Pixelle
python csv matplotlib graph
asked Nov 19 '18 at 11:27
PixellePixelle
102
add a comment |
1
I am encountering a very strange problem while processing data under Linux 16.04 using python 2.7.
I create a .csv file using this function :
from ast import literal_eval
with open('logs.csv') as f:
data = [literal_eval(line) for line in f]
the file is properly created and looks like that:
('2017-04-01 12:05:00','0.01770001','0.0177887','0.01780275','0.01770001')
('2017-04-01 12:10:00','0.0177887','0.01771308','0.01785263','0.01771039')
('2017-04-01 12:15:00','0.01773','0.01780092','0.01780092','0.01773')
('2017-04-01 12:20:00','0.0178','0.01781212','0.01784922','0.01774015')
('2017-04-01 12:25:00','0.01781212','0.01774528','0.01782994','0.01774528')
('2017-04-01 12:30:00','0.01774529','0.0178732','0.01788145','0.01774509')
('2017-04-01 12:35:00','0.01788145','0.01793318','0.01793318','0.01788145')
('2017-04-01 12:40:00','0.01794','0.01780093','0.01799984','0.01780092')
('2017-04-01 12:45:00','0.01785694','0.01806699','0.01807519','0.01785694')
('2017-04-01 12:50:00','0.01807999','0.01819687','0.01827573','0.018027')
('2017-04-01 12:55:00','0.01819687','0.01825402','0.0184','0.01800011')
('2017-04-01 13:00:00','0.01822416','0.01830994','0.01835554','0.0181777')
('2017-04-01 13:05:00','0.01825415','0.01810171','0.01830986','0.01810008')
('2017-04-01 13:10:00','0.01810174','0.01818991','0.01818991','0.01810173')
('2017-04-01 13:15:00','0.01818991','0.01818002','0.01819687','0.01818001')
('2017-04-01 13:20:00','0.01818002','0.01821999','0.01822','0.01818001')
I then pass it throught this code to draw a graph :
import numpy as np
import matplotlib.pyplot as plt
from matplotlib import dates, ticker
import matplotlib as mpl
from mpl_finance import candlestick_ohlc
from ast import literal_eval
mpl.style.use('default')
data =
ohlc_data =
with open('logsXMR.csv') as f:
data = [literal_eval(line) for line in f]
for line in data:
#ohlc_data.append((np.float64(line[0]), np.float64(line[1]), np.float64(line[2]), np.float64(line[3]), np.float64(line[4])))
ohlc_data.append((dates.datestr2num(line[0]), np.float64(line[1]), np.float64(line[2]), np.float64(line[3]), np.float64(line[4])))
fig, ax1 = plt.subplots()
candlestick_ohlc(ax1, ohlc_data, width = 0.5/((24*60)/5), colorup = 'g', colordown = 'r', alpha = 0.8)
#ax1.xaxis.set_major_formatter(dates.DateFormatter('%d/%m/%Y %H:%M'))
ax1.xaxis.set_major_locator(ticker.MaxNLocator(10))
plt.xticks(rotation = 30)
plt.grid()
plt.xlabel('Date')
plt.ylabel('Price')
plt.title('Historical Data XMRUSD')
plt.tight_layout()
plt.show()
But each time I get that error:
Traceback (most recent call last):
File "CSVing.py", line 15, in <module>
data = [literal_eval(line) for line in f]
File "/usr/lib/python2.7/ast.py", line 49, in literal_eval
node_or_string = parse(node_or_string, mode='eval')
File "/usr/lib/python2.7/ast.py", line 37, in parse
return compile(source, filename, mode, PyCF_ONLY_AST)
File "<unknown>", line 2
('2017-04-01 12:10:00','0.0177887','0.01771308','0.01785263','0.01771039')
^
I don't understand why I get this error because if I simply copy and paste my datas into another file, everything works fine, I can draw my graph flawlessly. I just don't get it because the 2 data files are identicals, no added space or anything.
what can cause this error and how can I proceed to be able to used my generated data file directly without the need to copy paste the datas in another file ?
thanks in advance,
Pixelle
python csv matplotlib graph
asked Nov 19 '18 at 11:27
PixellePixelle
102
Can you motivate why you useliteral_evalto create a text file? Useful ways to do such things arecsvorpandas.
– ImportanceOfBeingErnest
Nov 19 '18 at 11:51
yes sure, it was the subject of my previous post, it was easier that way: stackoverflow.com/questions/52976469/…
– Pixelle
Nov 19 '18 at 12:03
You didn't answer this comment though.
– ImportanceOfBeingErnest
Nov 19 '18 at 12:05
you're are right, I guess I'm not quite familiar with Stack Overflow yet. I do have controle over the whole code but the graph part came from an example found on the internet so since it was working fine, I kept the code as it was and adapted my datas format.
– Pixelle
Nov 19 '18 at 12:47
add a comment |
1
1
1
I am encountering a very strange problem while processing data under Linux 16.04 using python 2.7.
I create a .csv file using this function :
from ast import literal_eval
with open('logs.csv') as f:
data = [literal_eval(line) for line in f]
the file is properly created and looks like that:
('2017-04-01 12:05:00','0.01770001','0.0177887','0.01780275','0.01770001')
('2017-04-01 12:10:00','0.0177887','0.01771308','0.01785263','0.01771039')
('2017-04-01 12:15:00','0.01773','0.01780092','0.01780092','0.01773')
('2017-04-01 12:20:00','0.0178','0.01781212','0.01784922','0.01774015')
('2017-04-01 12:25:00','0.01781212','0.01774528','0.01782994','0.01774528')
('2017-04-01 12:30:00','0.01774529','0.0178732','0.01788145','0.01774509')
('2017-04-01 12:35:00','0.01788145','0.01793318','0.01793318','0.01788145')
('2017-04-01 12:40:00','0.01794','0.01780093','0.01799984','0.01780092')
('2017-04-01 12:45:00','0.01785694','0.01806699','0.01807519','0.01785694')
('2017-04-01 12:50:00','0.01807999','0.01819687','0.01827573','0.018027')
('2017-04-01 12:55:00','0.01819687','0.01825402','0.0184','0.01800011')
('2017-04-01 13:00:00','0.01822416','0.01830994','0.01835554','0.0181777')
('2017-04-01 13:05:00','0.01825415','0.01810171','0.01830986','0.01810008')
('2017-04-01 13:10:00','0.01810174','0.01818991','0.01818991','0.01810173')
('2017-04-01 13:15:00','0.01818991','0.01818002','0.01819687','0.01818001')
('2017-04-01 13:20:00','0.01818002','0.01821999','0.01822','0.01818001')
I then pass it throught this code to draw a graph :
import numpy as np
import matplotlib.pyplot as plt
from matplotlib import dates, ticker
import matplotlib as mpl
from mpl_finance import candlestick_ohlc
from ast import literal_eval
mpl.style.use('default')
data =
ohlc_data =
with open('logsXMR.csv') as f:
data = [literal_eval(line) for line in f]
for line in data:
#ohlc_data.append((np.float64(line[0]), np.float64(line[1]), np.float64(line[2]), np.float64(line[3]), np.float64(line[4])))
ohlc_data.append((dates.datestr2num(line[0]), np.float64(line[1]), np.float64(line[2]), np.float64(line[3]), np.float64(line[4])))
fig, ax1 = plt.subplots()
candlestick_ohlc(ax1, ohlc_data, width = 0.5/((24*60)/5), colorup = 'g', colordown = 'r', alpha = 0.8)
#ax1.xaxis.set_major_formatter(dates.DateFormatter('%d/%m/%Y %H:%M'))
ax1.xaxis.set_major_locator(ticker.MaxNLocator(10))
plt.xticks(rotation = 30)
plt.grid()
plt.xlabel('Date')
plt.ylabel('Price')
plt.title('Historical Data XMRUSD')
plt.tight_layout()
plt.show()
But each time I get that error:
Traceback (most recent call last):
File "CSVing.py", line 15, in <module>
data = [literal_eval(line) for line in f]
File "/usr/lib/python2.7/ast.py", line 49, in literal_eval
node_or_string = parse(node_or_string, mode='eval')
File "/usr/lib/python2.7/ast.py", line 37, in parse
return compile(source, filename, mode, PyCF_ONLY_AST)
File "<unknown>", line 2
('2017-04-01 12:10:00','0.0177887','0.01771308','0.01785263','0.01771039')
^
I don't understand why I get this error because if I simply copy and paste my datas into another file, everything works fine, I can draw my graph flawlessly. I just don't get it because the 2 data files are identicals, no added space or anything.
what can cause this error and how can I proceed to be able to used my generated data file directly without the need to copy paste the datas in another file ?
thanks in advance,
Pixelle
python csv matplotlib graph
asked Nov 19 '18 at 11:27
PixellePixelle
102
I am encountering a very strange problem while processing data under Linux 16.04 using python 2.7.
I create a .csv file using this function :
from ast import literal_eval
with open('logs.csv') as f:
data = [literal_eval(line) for line in f]
the file is properly created and looks like that:
('2017-04-01 12:05:00','0.01770001','0.0177887','0.01780275','0.01770001')
('2017-04-01 12:10:00','0.0177887','0.01771308','0.01785263','0.01771039')
('2017-04-01 12:15:00','0.01773','0.01780092','0.01780092','0.01773')
('2017-04-01 12:20:00','0.0178','0.01781212','0.01784922','0.01774015')
('2017-04-01 12:25:00','0.01781212','0.01774528','0.01782994','0.01774528')
('2017-04-01 12:30:00','0.01774529','0.0178732','0.01788145','0.01774509')
('2017-04-01 12:35:00','0.01788145','0.01793318','0.01793318','0.01788145')
('2017-04-01 12:40:00','0.01794','0.01780093','0.01799984','0.01780092')
('2017-04-01 12:45:00','0.01785694','0.01806699','0.01807519','0.01785694')
('2017-04-01 12:50:00','0.01807999','0.01819687','0.01827573','0.018027')
('2017-04-01 12:55:00','0.01819687','0.01825402','0.0184','0.01800011')
('2017-04-01 13:00:00','0.01822416','0.01830994','0.01835554','0.0181777')
('2017-04-01 13:05:00','0.01825415','0.01810171','0.01830986','0.01810008')
('2017-04-01 13:10:00','0.01810174','0.01818991','0.01818991','0.01810173')
('2017-04-01 13:15:00','0.01818991','0.01818002','0.01819687','0.01818001')
('2017-04-01 13:20:00','0.01818002','0.01821999','0.01822','0.01818001')
I then pass it throught this code to draw a graph :
import numpy as np
import matplotlib.pyplot as plt
from matplotlib import dates, ticker
import matplotlib as mpl
from mpl_finance import candlestick_ohlc
from ast import literal_eval
mpl.style.use('default')
data =
ohlc_data =
with open('logsXMR.csv') as f:
data = [literal_eval(line) for line in f]
for line in data:
#ohlc_data.append((np.float64(line[0]), np.float64(line[1]), np.float64(line[2]), np.float64(line[3]), np.float64(line[4])))
ohlc_data.append((dates.datestr2num(line[0]), np.float64(line[1]), np.float64(line[2]), np.float64(line[3]), np.float64(line[4])))
fig, ax1 = plt.subplots()
candlestick_ohlc(ax1, ohlc_data, width = 0.5/((24*60)/5), colorup = 'g', colordown = 'r', alpha = 0.8)
#ax1.xaxis.set_major_formatter(dates.DateFormatter('%d/%m/%Y %H:%M'))
ax1.xaxis.set_major_locator(ticker.MaxNLocator(10))
plt.xticks(rotation = 30)
plt.grid()
plt.xlabel('Date')
plt.ylabel('Price')
plt.title('Historical Data XMRUSD')
plt.tight_layout()
plt.show()
But each time I get that error:
Traceback (most recent call last):
File "CSVing.py", line 15, in <module>
data = [literal_eval(line) for line in f]
File "/usr/lib/python2.7/ast.py", line 49, in literal_eval
node_or_string = parse(node_or_string, mode='eval')
File "/usr/lib/python2.7/ast.py", line 37, in parse
return compile(source, filename, mode, PyCF_ONLY_AST)
File "<unknown>", line 2
('2017-04-01 12:10:00','0.0177887','0.01771308','0.01785263','0.01771039')
^
I don't understand why I get this error because if I simply copy and paste my datas into another file, everything works fine, I can draw my graph flawlessly. I just don't get it because the 2 data files are identicals, no added space or anything.
what can cause this error and how can I proceed to be able to used my generated data file directly without the need to copy paste the datas in another file ?
thanks in advance,
Pixelle
python csv matplotlib graph
python csv matplotlib graph
asked Nov 19 '18 at 11:27
PixellePixelle
102
asked Nov 19 '18 at 11:27
PixellePixelle
102
asked Nov 19 '18 at 11:27
PixellePixelle
102
asked Nov 19 '18 at 11:27
PixellePixelle
102
asked Nov 19 '18 at 11:27
PixellePixelle
102
102
Can you motivate why you useliteral_evalto create a text file? Useful ways to do such things arecsvorpandas.
– ImportanceOfBeingErnest
Nov 19 '18 at 11:51
yes sure, it was the subject of my previous post, it was easier that way: stackoverflow.com/questions/52976469/…
– Pixelle
Nov 19 '18 at 12:03
You didn't answer this comment though.
– ImportanceOfBeingErnest
Nov 19 '18 at 12:05
you're are right, I guess I'm not quite familiar with Stack Overflow yet. I do have controle over the whole code but the graph part came from an example found on the internet so since it was working fine, I kept the code as it was and adapted my datas format.
– Pixelle
Nov 19 '18 at 12:47
add a comment |
Can you motivate why you useliteral_evalto create a text file? Useful ways to do such things arecsvorpandas.
– ImportanceOfBeingErnest
Nov 19 '18 at 11:51
yes sure, it was the subject of my previous post, it was easier that way: stackoverflow.com/questions/52976469/…
– Pixelle
Nov 19 '18 at 12:03
You didn't answer this comment though.
– ImportanceOfBeingErnest
Nov 19 '18 at 12:05
you're are right, I guess I'm not quite familiar with Stack Overflow yet. I do have controle over the whole code but the graph part came from an example found on the internet so since it was working fine, I kept the code as it was and adapted my datas format.
– Pixelle
Nov 19 '18 at 12:47
Can you motivate why you use
literal_eval to create a text file? Useful ways to do such things are csv or pandas.– ImportanceOfBeingErnest
Nov 19 '18 at 11:51
Can you motivate why you use
literal_eval to create a text file? Useful ways to do such things are csv or pandas.– ImportanceOfBeingErnest
Nov 19 '18 at 11:51
yes sure, it was the subject of my previous post, it was easier that way: stackoverflow.com/questions/52976469/…
– Pixelle
Nov 19 '18 at 12:03
yes sure, it was the subject of my previous post, it was easier that way: stackoverflow.com/questions/52976469/…
– Pixelle
Nov 19 '18 at 12:03
You didn't answer this comment though.
– ImportanceOfBeingErnest
Nov 19 '18 at 12:05
You didn't answer this comment though.
– ImportanceOfBeingErnest
Nov 19 '18 at 12:05
you're are right, I guess I'm not quite familiar with Stack Overflow yet. I do have controle over the whole code but the graph part came from an example found on the internet so since it was working fine, I kept the code as it was and adapted my datas format.
– Pixelle
Nov 19 '18 at 12:47
you're are right, I guess I'm not quite familiar with Stack Overflow yet. I do have controle over the whole code but the graph part came from an example found on the internet so since it was working fine, I kept the code as it was and adapted my datas format.
– Pixelle
Nov 19 '18 at 12:47
add a comment |
1 Answer
1
active
oldest
votes
0
I would recommend rethinking the data format you have. I don't know where the data comes from, but it would be reasonable to store it in a way that does not contain parantheses etc.
If you really need to work with this data format, you may still use e.g. pandas and sanitize the format, by removing the characters that aren't useful.
u = """('2017-04-01 12:05:00','0.01770001','0.0177887','0.01780275','0.01770001')
('2017-04-01 12:10:00','0.0177887','0.01771308','0.01785263','0.01771039')
('2017-04-01 12:15:00','0.01773','0.01780092','0.01780092','0.01773')
('2017-04-01 12:20:00','0.0178','0.01781212','0.01784922','0.01774015')
('2017-04-01 12:25:00','0.01781212','0.01774528','0.01782994','0.01774528')
('2017-04-01 12:30:00','0.01774529','0.0178732','0.01788145','0.01774509')
('2017-04-01 12:35:00','0.01788145','0.01793318','0.01793318','0.01788145')
('2017-04-01 12:40:00','0.01794','0.01780093','0.01799984','0.01780092')
('2017-04-01 12:45:00','0.01785694','0.01806699','0.01807519','0.01785694')
('2017-04-01 12:50:00','0.01807999','0.01819687','0.01827573','0.018027')
('2017-04-01 12:55:00','0.01819687','0.01825402','0.0184','0.01800011')
('2017-04-01 13:00:00','0.01822416','0.01830994','0.01835554','0.0181777')
('2017-04-01 13:05:00','0.01825415','0.01810171','0.01830986','0.01810008')
('2017-04-01 13:10:00','0.01810174','0.01818991','0.01818991','0.01810173')
('2017-04-01 13:15:00','0.01818991','0.01818002','0.01819687','0.01818001')
('2017-04-01 13:20:00','0.01818002','0.01821999','0.01822','0.01818001')"""
import io
import pandas as pd
import matplotlib.pyplot as plt
from matplotlib import dates
from mpl_finance import candlestick_ohlc
replace = {"(" : "", ")" : "", "'" : ""}
df = pd.read_csv(io.StringIO(u), sep=",", header=None).replace(replace, regex=True)
# use pd.read_csv("myfilename.txt", ...) here for your real file
df[0] = dates.datestr2num(df[0])
df.iloc[:,1:] = df.iloc[:,1:].astype(float)
fig, ax1 = plt.subplots()
candlestick_ohlc(ax1, df.values, width = 0.5/((24*60)/5),
colorup = 'g', colordown = 'r', alpha = 0.8)
ax1.xaxis.set_major_formatter(dates.DateFormatter('%d/%m/%Y %H:%M'))
ax1.xaxis.set_major_locator(dates.MinuteLocator((0,15,30,45)))
plt.xticks(rotation = 30)
plt.grid()
plt.xlabel('Date')
plt.ylabel('Price')
plt.title('Historical Data XMRUSD')
plt.tight_layout()
plt.show()
Note that the data also does not seem to be in Ohlc format, hence the strange looking graph. But since nothing is known about the data, you need to find out the correct order yourself.
answered Nov 19 '18 at 12:51
ImportanceOfBeingErnestImportanceOfBeingErnest
133k13145221
You were right, I integrated that into my main code and it work like a charm. thank you ! I should have used a more standard way of doing things from the beginning.
– Pixelle
Nov 19 '18 at 13:38
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53373652%2fdata-processing-error-solved-by-a-copy-paste%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
0
I would recommend rethinking the data format you have. I don't know where the data comes from, but it would be reasonable to store it in a way that does not contain parantheses etc.
If you really need to work with this data format, you may still use e.g. pandas and sanitize the format, by removing the characters that aren't useful.
u = """('2017-04-01 12:05:00','0.01770001','0.0177887','0.01780275','0.01770001')
('2017-04-01 12:10:00','0.0177887','0.01771308','0.01785263','0.01771039')
('2017-04-01 12:15:00','0.01773','0.01780092','0.01780092','0.01773')
('2017-04-01 12:20:00','0.0178','0.01781212','0.01784922','0.01774015')
('2017-04-01 12:25:00','0.01781212','0.01774528','0.01782994','0.01774528')
('2017-04-01 12:30:00','0.01774529','0.0178732','0.01788145','0.01774509')
('2017-04-01 12:35:00','0.01788145','0.01793318','0.01793318','0.01788145')
('2017-04-01 12:40:00','0.01794','0.01780093','0.01799984','0.01780092')
('2017-04-01 12:45:00','0.01785694','0.01806699','0.01807519','0.01785694')
('2017-04-01 12:50:00','0.01807999','0.01819687','0.01827573','0.018027')
('2017-04-01 12:55:00','0.01819687','0.01825402','0.0184','0.01800011')
('2017-04-01 13:00:00','0.01822416','0.01830994','0.01835554','0.0181777')
('2017-04-01 13:05:00','0.01825415','0.01810171','0.01830986','0.01810008')
('2017-04-01 13:10:00','0.01810174','0.01818991','0.01818991','0.01810173')
('2017-04-01 13:15:00','0.01818991','0.01818002','0.01819687','0.01818001')
('2017-04-01 13:20:00','0.01818002','0.01821999','0.01822','0.01818001')"""
import io
import pandas as pd
import matplotlib.pyplot as plt
from matplotlib import dates
from mpl_finance import candlestick_ohlc
replace = {"(" : "", ")" : "", "'" : ""}
df = pd.read_csv(io.StringIO(u), sep=",", header=None).replace(replace, regex=True)
# use pd.read_csv("myfilename.txt", ...) here for your real file
df[0] = dates.datestr2num(df[0])
df.iloc[:,1:] = df.iloc[:,1:].astype(float)
fig, ax1 = plt.subplots()
candlestick_ohlc(ax1, df.values, width = 0.5/((24*60)/5),
colorup = 'g', colordown = 'r', alpha = 0.8)
ax1.xaxis.set_major_formatter(dates.DateFormatter('%d/%m/%Y %H:%M'))
ax1.xaxis.set_major_locator(dates.MinuteLocator((0,15,30,45)))
plt.xticks(rotation = 30)
plt.grid()
plt.xlabel('Date')
plt.ylabel('Price')
plt.title('Historical Data XMRUSD')
plt.tight_layout()
plt.show()
Note that the data also does not seem to be in Ohlc format, hence the strange looking graph. But since nothing is known about the data, you need to find out the correct order yourself.
answered Nov 19 '18 at 12:51
ImportanceOfBeingErnestImportanceOfBeingErnest
133k13145221
You were right, I integrated that into my main code and it work like a charm. thank you ! I should have used a more standard way of doing things from the beginning.
– Pixelle
Nov 19 '18 at 13:38
add a comment |
0
I would recommend rethinking the data format you have. I don't know where the data comes from, but it would be reasonable to store it in a way that does not contain parantheses etc.
If you really need to work with this data format, you may still use e.g. pandas and sanitize the format, by removing the characters that aren't useful.
u = """('2017-04-01 12:05:00','0.01770001','0.0177887','0.01780275','0.01770001')
('2017-04-01 12:10:00','0.0177887','0.01771308','0.01785263','0.01771039')
('2017-04-01 12:15:00','0.01773','0.01780092','0.01780092','0.01773')
('2017-04-01 12:20:00','0.0178','0.01781212','0.01784922','0.01774015')
('2017-04-01 12:25:00','0.01781212','0.01774528','0.01782994','0.01774528')
('2017-04-01 12:30:00','0.01774529','0.0178732','0.01788145','0.01774509')
('2017-04-01 12:35:00','0.01788145','0.01793318','0.01793318','0.01788145')
('2017-04-01 12:40:00','0.01794','0.01780093','0.01799984','0.01780092')
('2017-04-01 12:45:00','0.01785694','0.01806699','0.01807519','0.01785694')
('2017-04-01 12:50:00','0.01807999','0.01819687','0.01827573','0.018027')
('2017-04-01 12:55:00','0.01819687','0.01825402','0.0184','0.01800011')
('2017-04-01 13:00:00','0.01822416','0.01830994','0.01835554','0.0181777')
('2017-04-01 13:05:00','0.01825415','0.01810171','0.01830986','0.01810008')
('2017-04-01 13:10:00','0.01810174','0.01818991','0.01818991','0.01810173')
('2017-04-01 13:15:00','0.01818991','0.01818002','0.01819687','0.01818001')
('2017-04-01 13:20:00','0.01818002','0.01821999','0.01822','0.01818001')"""
import io
import pandas as pd
import matplotlib.pyplot as plt
from matplotlib import dates
from mpl_finance import candlestick_ohlc
replace = {"(" : "", ")" : "", "'" : ""}
df = pd.read_csv(io.StringIO(u), sep=",", header=None).replace(replace, regex=True)
# use pd.read_csv("myfilename.txt", ...) here for your real file
df[0] = dates.datestr2num(df[0])
df.iloc[:,1:] = df.iloc[:,1:].astype(float)
fig, ax1 = plt.subplots()
candlestick_ohlc(ax1, df.values, width = 0.5/((24*60)/5),
colorup = 'g', colordown = 'r', alpha = 0.8)
ax1.xaxis.set_major_formatter(dates.DateFormatter('%d/%m/%Y %H:%M'))
ax1.xaxis.set_major_locator(dates.MinuteLocator((0,15,30,45)))
plt.xticks(rotation = 30)
plt.grid()
plt.xlabel('Date')
plt.ylabel('Price')
plt.title('Historical Data XMRUSD')
plt.tight_layout()
plt.show()
Note that the data also does not seem to be in Ohlc format, hence the strange looking graph. But since nothing is known about the data, you need to find out the correct order yourself.
answered Nov 19 '18 at 12:51
ImportanceOfBeingErnestImportanceOfBeingErnest
133k13145221
You were right, I integrated that into my main code and it work like a charm. thank you ! I should have used a more standard way of doing things from the beginning.
– Pixelle
Nov 19 '18 at 13:38
add a comment |
0
0
0
I would recommend rethinking the data format you have. I don't know where the data comes from, but it would be reasonable to store it in a way that does not contain parantheses etc.
If you really need to work with this data format, you may still use e.g. pandas and sanitize the format, by removing the characters that aren't useful.
u = """('2017-04-01 12:05:00','0.01770001','0.0177887','0.01780275','0.01770001')
('2017-04-01 12:10:00','0.0177887','0.01771308','0.01785263','0.01771039')
('2017-04-01 12:15:00','0.01773','0.01780092','0.01780092','0.01773')
('2017-04-01 12:20:00','0.0178','0.01781212','0.01784922','0.01774015')
('2017-04-01 12:25:00','0.01781212','0.01774528','0.01782994','0.01774528')
('2017-04-01 12:30:00','0.01774529','0.0178732','0.01788145','0.01774509')
('2017-04-01 12:35:00','0.01788145','0.01793318','0.01793318','0.01788145')
('2017-04-01 12:40:00','0.01794','0.01780093','0.01799984','0.01780092')
('2017-04-01 12:45:00','0.01785694','0.01806699','0.01807519','0.01785694')
('2017-04-01 12:50:00','0.01807999','0.01819687','0.01827573','0.018027')
('2017-04-01 12:55:00','0.01819687','0.01825402','0.0184','0.01800011')
('2017-04-01 13:00:00','0.01822416','0.01830994','0.01835554','0.0181777')
('2017-04-01 13:05:00','0.01825415','0.01810171','0.01830986','0.01810008')
('2017-04-01 13:10:00','0.01810174','0.01818991','0.01818991','0.01810173')
('2017-04-01 13:15:00','0.01818991','0.01818002','0.01819687','0.01818001')
('2017-04-01 13:20:00','0.01818002','0.01821999','0.01822','0.01818001')"""
import io
import pandas as pd
import matplotlib.pyplot as plt
from matplotlib import dates
from mpl_finance import candlestick_ohlc
replace = {"(" : "", ")" : "", "'" : ""}
df = pd.read_csv(io.StringIO(u), sep=",", header=None).replace(replace, regex=True)
# use pd.read_csv("myfilename.txt", ...) here for your real file
df[0] = dates.datestr2num(df[0])
df.iloc[:,1:] = df.iloc[:,1:].astype(float)
fig, ax1 = plt.subplots()
candlestick_ohlc(ax1, df.values, width = 0.5/((24*60)/5),
colorup = 'g', colordown = 'r', alpha = 0.8)
ax1.xaxis.set_major_formatter(dates.DateFormatter('%d/%m/%Y %H:%M'))
ax1.xaxis.set_major_locator(dates.MinuteLocator((0,15,30,45)))
plt.xticks(rotation = 30)
plt.grid()
plt.xlabel('Date')
plt.ylabel('Price')
plt.title('Historical Data XMRUSD')
plt.tight_layout()
plt.show()
Note that the data also does not seem to be in Ohlc format, hence the strange looking graph. But since nothing is known about the data, you need to find out the correct order yourself.
answered Nov 19 '18 at 12:51
ImportanceOfBeingErnestImportanceOfBeingErnest
133k13145221
I would recommend rethinking the data format you have. I don't know where the data comes from, but it would be reasonable to store it in a way that does not contain parantheses etc.
If you really need to work with this data format, you may still use e.g. pandas and sanitize the format, by removing the characters that aren't useful.
u = """('2017-04-01 12:05:00','0.01770001','0.0177887','0.01780275','0.01770001')
('2017-04-01 12:10:00','0.0177887','0.01771308','0.01785263','0.01771039')
('2017-04-01 12:15:00','0.01773','0.01780092','0.01780092','0.01773')
('2017-04-01 12:20:00','0.0178','0.01781212','0.01784922','0.01774015')
('2017-04-01 12:25:00','0.01781212','0.01774528','0.01782994','0.01774528')
('2017-04-01 12:30:00','0.01774529','0.0178732','0.01788145','0.01774509')
('2017-04-01 12:35:00','0.01788145','0.01793318','0.01793318','0.01788145')
('2017-04-01 12:40:00','0.01794','0.01780093','0.01799984','0.01780092')
('2017-04-01 12:45:00','0.01785694','0.01806699','0.01807519','0.01785694')
('2017-04-01 12:50:00','0.01807999','0.01819687','0.01827573','0.018027')
('2017-04-01 12:55:00','0.01819687','0.01825402','0.0184','0.01800011')
('2017-04-01 13:00:00','0.01822416','0.01830994','0.01835554','0.0181777')
('2017-04-01 13:05:00','0.01825415','0.01810171','0.01830986','0.01810008')
('2017-04-01 13:10:00','0.01810174','0.01818991','0.01818991','0.01810173')
('2017-04-01 13:15:00','0.01818991','0.01818002','0.01819687','0.01818001')
('2017-04-01 13:20:00','0.01818002','0.01821999','0.01822','0.01818001')"""
import io
import pandas as pd
import matplotlib.pyplot as plt
from matplotlib import dates
from mpl_finance import candlestick_ohlc
replace = {"(" : "", ")" : "", "'" : ""}
df = pd.read_csv(io.StringIO(u), sep=",", header=None).replace(replace, regex=True)
# use pd.read_csv("myfilename.txt", ...) here for your real file
df[0] = dates.datestr2num(df[0])
df.iloc[:,1:] = df.iloc[:,1:].astype(float)
fig, ax1 = plt.subplots()
candlestick_ohlc(ax1, df.values, width = 0.5/((24*60)/5),
colorup = 'g', colordown = 'r', alpha = 0.8)
ax1.xaxis.set_major_formatter(dates.DateFormatter('%d/%m/%Y %H:%M'))
ax1.xaxis.set_major_locator(dates.MinuteLocator((0,15,30,45)))
plt.xticks(rotation = 30)
plt.grid()
plt.xlabel('Date')
plt.ylabel('Price')
plt.title('Historical Data XMRUSD')
plt.tight_layout()
plt.show()
Note that the data also does not seem to be in Ohlc format, hence the strange looking graph. But since nothing is known about the data, you need to find out the correct order yourself.
answered Nov 19 '18 at 12:51
ImportanceOfBeingErnestImportanceOfBeingErnest
133k13145221
answered Nov 19 '18 at 12:51
ImportanceOfBeingErnestImportanceOfBeingErnest
133k13145221
answered Nov 19 '18 at 12:51
ImportanceOfBeingErnestImportanceOfBeingErnest
133k13145221
answered Nov 19 '18 at 12:51
ImportanceOfBeingErnestImportanceOfBeingErnest
133k13145221
133k13145221
You were right, I integrated that into my main code and it work like a charm. thank you ! I should have used a more standard way of doing things from the beginning.
– Pixelle
Nov 19 '18 at 13:38
add a comment |
You were right, I integrated that into my main code and it work like a charm. thank you ! I should have used a more standard way of doing things from the beginning.
– Pixelle
Nov 19 '18 at 13:38
You were right, I integrated that into my main code and it work like a charm. thank you ! I should have used a more standard way of doing things from the beginning.
– Pixelle
Nov 19 '18 at 13:38
You were right, I integrated that into my main code and it work like a charm. thank you ! I should have used a more standard way of doing things from the beginning.
– Pixelle
Nov 19 '18 at 13:38
add a comment |
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53373652%2fdata-processing-error-solved-by-a-copy-paste%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Can you motivate why you use
literal_evalto create a text file? Useful ways to do such things arecsvorpandas.– ImportanceOfBeingErnest
Nov 19 '18 at 11:51
yes sure, it was the subject of my previous post, it was easier that way: stackoverflow.com/questions/52976469/…
– Pixelle
Nov 19 '18 at 12:03
You didn't answer this comment though.
– ImportanceOfBeingErnest
Nov 19 '18 at 12:05
you're are right, I guess I'm not quite familiar with Stack Overflow yet. I do have controle over the whole code but the graph part came from an example found on the internet so since it was working fine, I kept the code as it was and adapted my datas format.
– Pixelle
Nov 19 '18 at 12:47