Writing a list to a file with Python, with newlines
do note that writelines doesn’t add newlines because it mirrors readlines , which also doesn’t remove them.
26 Answers 26
with open('your_file.txt', 'w') as f: for line in lines: f.write(f"\n") with open('your_file.txt', 'w') as f: for line in lines: f.write("%s\n" % line) For Python 2, one may also use:
with open('your_file.txt', 'w') as f: for line in lines: print >> f, line If you’re keen on a single function call, at least remove the square brackets [] , so that the strings to be printed get made one at a time (a genexp rather than a listcomp) — no reason to take up all the memory required to materialize the whole list of strings.
What are you going to do with the file? Does this file exist for humans, or other programs with clear interoperability requirements?
If you are just trying to serialize a list to disk for later use by the same python app, you should be pickleing the list.
import pickle with open('outfile', 'wb') as fp: pickle.dump(itemlist, fp) with open ('outfile', 'rb') as fp: itemlist = pickle.load(fp) «Warning: The pickle module is not secure. Only unpickle data you trust. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling. Never unpickle data that could have come from an untrusted source, or that could have been tampered with.» — From the same manual page linked on the answer.
with open("outfile", "w") as outfile: outfile.write("\n".join(itemlist)) To ensure that all items in the item list are strings, use a generator expression:
with open("outfile", "w") as outfile: outfile.write("\n".join(str(item) for item in itemlist)) Remember that itemlist takes up memory, so take care about the memory consumption.
Using Python 3 and Python 2.6+ syntax:
with open(filepath, 'w') as file_handler: for item in the_list: file_handler.write("<>\n".format(item)) This is platform-independent. It also terminates the final line with a newline character, which is a UNIX best practice.
Starting with Python 3.6, «<>\n».format(item) can be replaced with an f-string: f»\n» .
Yet another way. Serialize to json using simplejson (included as json in python 2.6):
>>> import simplejson >>> f = open('output.txt', 'w') >>> simplejson.dump([1,2,3,4], f) >>> f.close() This is useful because the syntax is pythonic, it’s human readable, and it can be read by other programs in other languages.
I thought it would be interesting to explore the benefits of using a genexp, so here’s my take.
The example in the question uses square brackets to create a temporary list, and so is equivalent to:
file.writelines( list( "%s\n" % item for item in list ) ) Which needlessly constructs a temporary list of all the lines that will be written out, this may consume significant amounts of memory depending on the size of your list and how verbose the output of str(item) is.
Drop the square brackets (equivalent to removing the wrapping list() call above) will instead pass a temporary generator to file.writelines() :
file.writelines( "%s\n" % item for item in list ) This generator will create newline-terminated representation of your item objects on-demand (i.e. as they are written out). This is nice for a couple of reasons:
- Memory overheads are small, even for very large lists
- If str(item) is slow there’s visible progress in the file as each item is processed
This avoids memory issues, such as:
In [1]: import os In [2]: f = file(os.devnull, "w") In [3]: %timeit f.writelines( "%s\n" % item for item in xrange(2**20) ) 1 loops, best of 3: 385 ms per loop In [4]: %timeit f.writelines( ["%s\n" % item for item in xrange(2**20)] ) ERROR: Internal Python error in the inspect module. Below is the traceback from this internal error. Traceback (most recent call last): . MemoryError (I triggered this error by limiting Python’s max. virtual memory to ~100MB with ulimit -v 102400 ).
Putting memory usage to one side, this method isn’t actually any faster than the original:
In [4]: %timeit f.writelines( "%s\n" % item for item in xrange(2**20) ) 1 loops, best of 3: 370 ms per loop In [5]: %timeit f.writelines( ["%s\n" % item for item in xrange(2**20)] ) 1 loops, best of 3: 360 ms per loop How can I write list of strings to file, adding newlines?
Can you please add a sample how you want the output to look? It’s not exactly clear where you want to add the newlines.
9 Answers 9
This is complicated, compared to a simple print , which is designed to do just what the original poster asked.
@Ignacio: I would indeed not say that data.write(‘\n’) is «wrong». However, I do think that it goes against the Zen of Python («There should be one— and preferably only one —obvious way to do it.»): since print is designed to add a newline, it feels really strange to me not to use it in this case.
@EOL: I disagree that print is the «obvious» way here. In order to use print with a file handle under Python 2.x, you have to know the redirect operator >> which never gets used in any other place in the language. The print-chevron is just syntactic sugar for write() anyways. file.write() is designed for getting your (simple) data into a file, irrespective of those data’s contents. In Python 3, I could perhaps agree with you.
@Josh: Fair enough. 🙂 On the other hand, the chevron operator in Python 2.x is a must when you have lots of newline-separated lines to print to a file: doing print >> …, … is much lighter than doing a lot of ….write(«%s\n» % …) . It is also arguably more legible, as it means «I’m printing lines with newlines», whereas the format string in write() has to be visually parsed before coming to this conclusion.
Don’t know what you mean by «lighter». I do see and cede your point about legibility, however; that’s a good way to look at it.
A properly-placed data.write(‘\n’) will handle that. Just indent it appropriately for the loop you want to punctuate.
As other answers gave already pointed out, you can do it by appending a ‘\n’ to c+n or by using the format string «%s%s\n».
Just as a matter of interest, I think it would be more Pythonic to use a list comprehension instead of two nested loops:
data.write("\n".join("%s%s"%(c, n) for c in s_chars for n in nums)) @detly: Yes it does. In this case the string generated should be short enough. It should be somewhat faster as long as there is enough memory.
@detly: I added an answer with the same basic behavior that avoids that peak memory usage issue precisely to solve that. Too few people know about writelines . My answer uses additional builtins to push all the work to C, but this answer could still get the benefit of peak memory usage reduction with the small tweak: data.writelines(‘%s%s\n’ % (c, n) for c in s_chars for n in nums)
Python’s print is the standard «print with newline» function.
Therefore, you can directly do, if you use Python 2.x:
Pushing more work to the C layer with writelines and product :
from future_builtins import map # Only do this on Python 2; makes map generator function import itertools def generator(): nums = ['09', '98', '87', '76', '65', '54', '43'] s_chars = ['*', '&', '^', '%', '$', '#', '@',] # Append newlines up front, to avoid doing the work len(nums) * len(s_chars) times # product will realize list from generator internally and discard values when done nums_newlined = (n + "\n" for s in nums) with open("list.txt", "w") as data: data.writelines(map(''.join, itertools.product(s_chars, nums_newlined))) This produces the same effect as the nested loop, but does so with builtins implemented in C (in the CPython reference interpreter anyway), removing byte code execution overhead from the picture; this can dramatically improve performance, particularly for larger inputs, and unlike other solutions involving ‘\n’.join of the whole output into a single string to perform a single write call, it’s iterating as it writes, so peak memory usage remains fixed instead of requiring you to realize the entire output in memory all at once in a single string.
Save a list to a .txt file
Is there a function in python that allows us to save a list in a txt file and keep its format? If I have the list:
5 Answers 5
values = ['1', '2', '3'] with open("file.txt", "w") as output: output.write(str(values)) If you have more then 1 dimension array
with open("file.txt", 'w') as output: for row in values: output.write(str(row) + '\n') Code to write without ‘[‘ and ‘]’
with open("file.txt", 'w') as file: for row in values: s = " ".join(map(str, row)) file.write(s+'\n') +1 this one is the most parse-able way to write a list into a file, the other answer is also correct of course but will give a strange python list format in the output file
You can use inbuilt library pickle
This library allows you to save any object in python to a file
This library will maintain the format as well
import pickle with open('/content/list_1.ob', 'wb') as fp: pickle.dump(list_1, fp) you can also read the list back as an object using same library
with open ('/content/list_1.ob', 'rb') as fp: list_1 = pickle.load(fp) NOTE:: File can have any extension you are comfortable with. These files are binary and are not supposed to be viewed manually.
Aren’t pickle files a binary format? I got the impression, from teh .txt extension on the OP’s file that he wanted a plain-text file that he could view, say, in a text viewer. Maybe I am wrong about pickle?
@Brian .. yes We can’t open the pickled file and view it in plain text. .txt may give this feeling. I am editing my answer accordingly.
For performance and other purposes, and assuming you will also want to read and write stuff from a file, use json formats!
P.S. — You can write to txts as well, but jsons are a standard for saving such objects!
ORJSON is the goto standard for this task. Also supports nested lists/dicts/other complex structures.
NOTE : ORJSON is faster and more serializable than the inbuilt JSON library but both can perform the same.
import orjson def pretty_view_dict(normal_dict): print(orjson.dumps(normal_dict, option=orjson.OPT_INDENT_2).decode('utf-8')) def read_json_file(fpath): with open(fpath, "r") as f: data = orjson.loads(f.read()) return data def write_json_file(fpath, data): with open(fpath, "wb") as f: f.write(orjson.dumps(data, option= orjson.OPT_INDENT_2)) return True I use a logger of my own creation :
import json import timeit import traceback import sys import unidecode def main_writer(f,argument): try: f.write(str(argument)) except UnicodeEncodeError: f.write(unidecode.unidecode(argument)) def logger(*argv,logfile="log.txt",singleLine = False): """ Writes Logs to LogFile """ with open(logfile, 'a+') as f: for arg in argv: if arg == "<>": continue if type(arg) == dict and len(arg)!=0: json_object = json.dumps(arg, indent=4, default=str) f.write(str(json_object)) f.flush() """ for key,val in arg.items(): f.write(str(key) + " : "+ str(val)) f.flush() """ elif type(arg) == list and len(arg)!=0: for each in arg: main_writer(f,each) f.write("\n") f.flush() else: main_writer(f,arg) f.flush() if singleLine==False: f.write("\n") if singleLine==True: f.write("\n") def tryFunc(func, func_name=None, *args, **kwargs): """ Time for Successfull Runs Exception Traceback for Unsuccessful Runs """ stack = traceback.extract_stack() filename, codeline, funcName, text = stack[-2] func_name = func.__name__ if func_name is None else func_name # sys._getframe().f_code.co_name # func.__name__ start = timeit.default_timer() x = None try: x = func(*args, **kwargs) stop = timeit.default_timer() # logger("Time to Run <> : <>".format(func_name, stop - start)) except Exception as e: logger("Exception Occurred for <> :".format(func_name)) logger("Basic Error Info :",e) logger("Full Error TraceBack :") # logger(e.message, e.args) logger(traceback.format_exc()) return x def bad_func(): return 'a'+ 7 if __name__ == '__main__': logger(234) logger([1,2,3]) logger(['a','b','c']) logger() tryFunc(bad_func)