• Register
1 vote
105 views

Problem:

Hello everyone,

I am trying to create a webscraper which will take links from CSV file and scrape data.

but whenever I run this code it says the object of nonetype has no len().

Below is my code:

import requests
from bs4 import BeautifulSoup
import time
import os
import csv
 
file = {}
final_data = []
 
def read_file():
    global file
    f = open("Data.csv")
    for line in f.readlines():
        data = line.split(",")
        file[data[0]]= data[1]
 
def writedata(alldata1, filename):
    with open("./"+filename, "w")as csvfile:
        csvfile = csv.writer(csvfile, delimiter=",")
        csvfile.writerow("")
        for i in range(0, len(alldata1)):
            csvfile.writerow(alldata1[i])
 
def parse_data():
    global file
    global final_data
    for data_obj in file.keys():
        link = file[data_obj]
        #print(link)
        data = getbyget(link, {})
        soup = BeautifulSoup(data, "html.parser")
        get_details = soup.find_all(class_="box2-body")[0]
        details = get_details.find_all(id="ctl00_ContentPlaceHolder1_Lblproposedenddt")
        name = get_details.find_all(id="ctl00_ContentPlaceHolder1_lblprojectname")
        date = ""
        names = ""
        sublist = []
        for more in details:
            date = more.text
            sublist.append(date)
        for nme in name:
            names = nme.text
            sublist.append(names)
 
        get_state = soup.find_all(class_="box2-body")[1]
        state = get_state.find_all(id="ctl00_ContentPlaceHolder1_lblState")
        city = get_state.find_all(id="ctl00_ContentPlaceHolder1_lbldistrict")
        cities = ""
        states = ""
        for statename in state:
            states = statename.text
            sublist.append(states)
        for citys in city:
            cities = citys.text
            sublist.append(cities)
             
        get_pmptername = soup.find_all(class_="box2-body")[2]
        promotername = get_pmptername.find_all(id="ctl00_ContentPlaceHolder1_Lblpromotername")
        geta = get_pmptername.find_all("a")
        alinks = ""
        promote = ""
        for promoter in promotername:
            promote = promoter.text
            sublist.append(promote)
        for linka in geta:
            alinks = "http://up-rera.in/"+ linka["href"]
            sublist.append(alinks)
            final_data.append(sublist)
            print(final_data)
 
def getbyget(url, values):
    res = requests.get(url, data=values)
    data = res.text
    return data
 
def main():
    read_file()
    datas = parse_data()
    writedata(datas, "UP_DATA.csv")                                                      
main()

What’s going wrong here?

12 7 7
15,250 points

Please log in or register to answer this question.

1 Answer

1 vote

Solution:

Most of the time, a TypeError involving the None object indicates that a function mistakenly returned None because of a missing return statement. As per your program, there is no return statement in parse_data(), which means that datas has value None.

13 9 6
94,240 points

Related questions

0 votes
1 answer 39 views
39 views
Problem: I'm seeing weird behavior on this code: images = dict(cover=[],second_row=[],additional_rows=[]) for pic in pictures: if len(images['cover']) == 0: images['cover'] = pic.path_thumb_l elif len(images['second_row']) < 3: images['second_row'].append(pic.path_thumb_m) ... == 0: TypeError: object of type 'NoneType' has no len() I can't figure out what's wrong in this. Maybe some scope issue?
asked Jan 10 Mashhoodch 13k points
0 votes
1 answer 40 views
40 views
Problem: How to I Solve this : typeerror: object of type 'map' has no len() HELP! I need help understanding this: typeerror: object of type 'map' has no len() Can someone please help solve this, I am new in python().
asked Mar 2 Mashhoodch 13k points
0 votes
1 answer 29 views
29 views
I am creating a cookie clicker game, where there is a surface that displays how many cookies I have. Here is my code for drawing text. def draw_text(self, text, font_name, size, color, x, y, align="nw"): font = pg.font.Font(font_name, size) text_surface = ... does self.draw_text('Cookies: {}'.format(len(self.cookie_count)) give me an error? How come the length of self.cookie_count is not printed?
asked Oct 25, 2020 psandprop 2.4k points
0 votes
1 answer 46 views
46 views
Problem: From Python: Attribute Error - 'NoneType' object has no attribute 'something': NoneType what means that instead of an instance of whatever Class or Object you think you're working with? you've actually got None. what That usually means that an assignment or function call up above failed or returned an unexpected result?
asked Jan 27 Mashhoodch 13k points
0 votes
1 answer 21 views
21 views
Problem: I use Regex to retrieve certain content from a search box on a webpage with selenium.webDriver. searchbox = driver.find_element_by_class_name("searchbox") searchbox_result = re.match(r"^.*(?=(\())", searchbox).group() The code works as long as the search ... get error: AttributeError: 'NoneType' object has no attribute 'group' How can I make the script handle the "No results" situation?
asked Jan 9 Mashhoodch 13k points
0 votes
1 answer 153 views
153 views
Using Python Regex to retrieve certain content from a search box on a webpage with selenium.webDriver searchbox = driver.find_element_by_class_name("searchbox") searchbox_result = re.match(r"^.*(?=(\())", searchbox).group() The code works as long as the search box ... is error: AttributeError: 'NoneType' object has no attribute 'group' How can I make the script handle the "No results" situation?
asked Oct 24, 2020 psandprop 2.4k points
0 votes
1 answer 5 views
0 votes
1 answer 15 views
15 views
New to programming and am unsure why I am getting this error count=int(input ("How many donuts do you have?")) if count <= 10: print ("number of donuts: " ) +str(count) else: print ("Number of donuts: many")
asked Oct 25, 2020 psandprop 2.4k points
0 votes
1 answer 2 views
2 views
Problem: I'm following a tutorial on neural nets1 It's in Python 2.7. I'm using 3.4. This is the line that troubles me: if test_data: n_test = len(test_data) I get: TypeError: object of type 'zip' has no len(). Is there a way to rewrite it so that it works in 3.4?
asked 4 days ago Atik03 10.1k points