What are generators in Python?

Generators in Python are the function that returns an object (iterator) which we can iterate over (one value at a time).

Generator removes overhead in building an iterator in python and is short and non counter intuitive. Generator handles all the overhead occurred in python.

A function is differed from a generator by the use of word yield instead of return as used for a function (def). Both yield and return will return some value from a function. The difference between a yield and return is that return statement terminates a function entirely, yield statement pauses the function saving all its states and later continues from there for further calls.

Generators are divided in to two parts:

1. Generator Function

Generator function is like a normal function but it generates a value using yield rather than return.

Generator Function Example

# generator function
def simple_generator():
    yield 1
    yield 2
    yield 3

# Iterating simple generator
print("Values retrieved from simple generator are: ")
for value in simple_generator():
    print(value)

# Now let's check type of simple_generator
print('Type of simple generator is: ', type(simple_generator))

The output of the above program is:

Values retrieved from simple generator are: 
1
2
3
Type of simple generator is:  <class 'function'>

2. Generator Object

Generator object returns a generator object. Generator objects are used either by calling the next method on generator object or using the generator object in a for loop.

Generator Object Examaple

# generator function
def generator_obj():
    yield 1
    yield 2
    yield 3
    
# generator object
obj = generator_obj()

# itertaion over generator objects using next
print("Using next: ")
print(next(obj))
print(next(obj))
print(next(obj))

print('Type of generator_obj is: ', type(generator_obj))
print('Type of obj is: ', type(obj))

The output of the above program is:

Using next: 
1
2
3
Type of generator_obj is:  <class 'function'>
Type of obj is:  <class 'generator'>

Real Life Scenario of Python Generator

A real life example of generator is to read a large text file. The code is given as follows:

# opening file in read mode
def read_file(file_name):
    text_file = open(file_name,r)
    
    # while file is present
    while True:
        line_data = text_file.readline()
        
        # read the file lines
        if not line_data():
        # if no lines to read close file
            text_file.close()
            break
        # data to display
        yield line_data
        

# passing a file to be read
file_data = read_file(file_form_system)

# iterate and display
for i in file_data:
    print(i)