Site
Home / Forum / 3rd Party Projects / [Plex Add-on] Sports Scraper

[Plex Add-on] Sports Scraper


Posted: 10 Feb 2022 11:23

zag
Posts: 3,327
Joined: 2020-03-23

Yes they work, I would need to see a log though. It should be pretty simple to diagnose.

Posted: 11 Feb 2022 23:15
sport1234
Posts: 20
Joined: 2022-02-08

Ok zag. Thank you for confirming that they work, and the quick reply. I am not sure how to get the log file, but I will figure it out, do a metadata refresh, and post the log on here, so you can review, and hopefully help me out to get this proof of concept to work, so I can apply it to other sports. Thanks again.

Posted: 11 Feb 2022 23:49
sport1234
Posts: 20
Joined: 2022-02-08

Hi Zag. I have attached my latest scan for my test file from the sports database.

Feb 12, 2022 01:42:03.585 [17272] INFO - Windows version: 6.2 (Build 9200), language en-US
Feb 12, 2022 01:42:03.585 [17272] INFO - 8 1800 MHz processor(s): Architecture=0, Level=6, Revision=36363 Processor Identifier=Intel64 Family 6 Model 142 Stepping 11, GenuineIntel
Feb 12, 2022 01:42:03.585 [17272] DEBUG - "C:\Program Files (x86)\Plex\Plex Media Server\Plex Media Scanner.exe" --scan --refresh --force --section 1 --activity fb662e1e-cb4b-4e78-9f93-97f1afae3de0
Feb 12, 2022 01:42:03.580 [21472] DEBUG - Opening 20 database sessions to library (com.plexapp.plugins.library), SQLite 3.35.5, threadsafe=1
Feb 12, 2022 01:42:04.343 [21472] DEBUG - HTTP requesting PUT https://127.0.0.1:32400/activities/fb662e1e-cb4b-4e78-9f93-97f1afae3de0?percentComplete=0
Feb 12, 2022 01:42:04.352 [15792] DEBUG - HTTP requesting GET https://127.0.0.1:32400/metadata/notify/cullTimeline?librarySectionID=1&sinceTime=1644621724
Feb 12, 2022 01:42:04.374 [3580] DEBUG - [HttpClient] HTTP/1.1 (0.0s) 200 response from PUT https://127.0.0.1:32400/activities/fb662e1e-cb4b-4e78-9f93-97f1afae3de0?percentComplete=0
Feb 12, 2022 01:42:04.400 [3580] DEBUG - [HttpClient] HTTP/1.1 (0.0s) 200 response from GET https://127.0.0.1:32400/metadata/notify/cullTimeline?librarySectionID=1&sinceTime=1644621724
Feb 12, 2022 01:42:04.408 [21472] DEBUG - Scanning Sports using en(SportScanner) with 0 current media items and 1 section locations in the database.
Feb 12, 2022 01:42:05.185 [21472] DEBUG - Performing a scan with 'SportScanner' (language: en virtual: 0).
Feb 12, 2022 01:42:05.185 [21472] DEBUG - * Scanning M:\Sport\Fighting\UFC\2021
Feb 12, 2022 01:42:05.215 [21472] DEBUG - Scanner: Processing directory M:\Sport\Fighting\UFC\2021 (parent: no)
Feb 12, 2022 01:42:05.299 [21472] DEBUG - Directory had 1 files, database had 0 files, can't skip.
Feb 12, 2022 01:42:05.316 [21472] DEBUG - HTTP requesting PUT https://127.0.0.1:32400/activities/fb662e1e-cb4b-4e78-9f93-97f1afae3de0?percentComplete=%2B99
Feb 12, 2022 01:42:05.317 [3580] DEBUG - [HttpClient] HTTP/1.1 (0.0s) 200 response from PUT https://127.0.0.1:32400/activities/fb662e1e-cb4b-4e78-9f93-97f1afae3de0?percentComplete=%2B99 (reused)
Feb 12, 2022 01:42:05.319 [21472] DEBUG - HTTP requesting PUT https://127.0.0.1:32400/activities/fb662e1e-cb4b-4e78-9f93-97f1afae3de0?percentComplete=99.000000
Feb 12, 2022 01:42:05.320 [3580] DEBUG - [HttpClient] HTTP/1.1 (0.0s) 200 response from PUT https://127.0.0.1:32400/activities/fb662e1e-cb4b-4e78-9f93-97f1afae3de0?percentComplete=99.000000 (reused)
Feb 12, 2022 01:42:05.320 [21472] DEBUG - Removing 0 media items that were left.
Feb 12, 2022 01:42:05.320 [21472] DEBUG - Removing 0 directories that were left.
Feb 12, 2022 01:42:05.321 [21472] DEBUG - HTTP requesting PUT https://127.0.0.1:32400/activities/fb662e1e-cb4b-4e78-9f93-97f1afae3de0?percentComplete=100
Feb 12, 2022 01:42:05.323 [3580] DEBUG - [HttpClient] HTTP/1.1 (0.0s) 200 response from PUT https://127.0.0.1:32400/activities/fb662e1e-cb4b-4e78-9f93-97f1afae3de0?percentComplete=100 (reused)
Feb 12, 2022 01:42:05.324 [21472] DEBUG - Refreshing section 1 of type: 2
Feb 12, 2022 01:42:05.324 [4828] DEBUG - Refreshing 0 IDs.

Posted: 12 Feb 2022 09:03

zag
Posts: 3,327
Joined: 2020-03-23

Hard to tell really as the only info in that log is the folder structure which looks ok to me.

M:\Sport\Fighting\UFC\2021

Not sure what else to suggest, I know it works on Kodi.

Posted: 12 Feb 2022 10:28
sport1234
Posts: 20
Joined: 2022-02-08

Zag, I saw that as well. I also saw that it finds the directory, it finds the only file in the directory, but it does not match with the sportsdb. It seems an issue of how I named it(which I got the name exactly from the particular event from the sportsdb), or the Plex python wrapper does not work.. You use this particular wrapper for Plex metadata, or do you use Kodi for sports metadata? Should I look into using Plex for Kodi, and for sports, Kodi directly? As always, thank you for your help…

Posted: 12 Feb 2022 11:08
sport1234
Posts: 20
Joined: 2022-02-08

Zag, would really help, is if you are using the Plex scripts for sports metadata, and if you can send me a sample directory structure, with particular filename structure that you are using, so I can compare and see any differences..

Posted: 12 Feb 2022 13:19

zag
Posts: 3,327
Joined: 2020-03-23

This is for the Kodi scraper, but it should be the same I believe

https://www.thesportsdb.com/Sports_fake_video_files.zip

Posted: 12 Feb 2022 19:26
sport1234
Posts: 20
Joined: 2022-02-08

Hi Zag. Thank you for the information. I followed your naming convention, and I see how you got this from the Sportsdb, but I am still having issue with it recognizing the file. Let me play with this for a bit, and see if I can get it to work.



Posted: 14 Feb 2022 15:02
jbalcorn
Posts: 20
Joined: 2022-01-24

I've been working on the SportScanner - I keep an eye on the Issues at https://github.com/mmmmmtasty/SportScanner/issues and the owner has been good about merging my pull requests, so if you see anything wrong report it there.

Posted: 14 Feb 2022 15:10
jbalcorn
Posts: 20
Joined: 2022-01-24

Example Sports Directory for Plex:

https://github.com/mmmmmtasty/SportScanner/wiki/Example-Sports-directory-for-Plex

zag
Posted: 14 Feb 2022 21:12
sport1234
Posts: 20
Joined: 2022-02-08

Hi there. Thank you for letting me know about where to post it ask questions. This is great. I appreciate your help with this

Posted: 14 Feb 2022 21:46
sport1234
Posts: 20
Joined: 2022-02-08

Hi, I started looking at your latest email, and it is very helpful for naming convention. My question is, what does it mean when you have something like:

File name.Sportscanner?

Also, do they have to be .mp4 files for the sport scanner to recognize them and get the metadata?

I use Happauge PVR and by default it creates .TS files, so most of my files are in that format, even though more recently, I have since figured out how to record directly in .mp4 files.

Thank you again for your help

Posted: 14 Feb 2022 21:49
sport1234
Posts: 20
Joined: 2022-02-08

One more question. What should be in the in the SportScanner.txt files, as I saw that you have them in every directory, or maybe a sample may be helpful.

Posted: 15 Feb 2022 19:04
jbalcorn
Posts: 20
Joined: 2022-01-24

"Additional Metadata"

https://github.com/mmmmmtasty/SportScanner#additional-metadata

SportScanner.txt has either
xxyy 7,1
or
xxxx

*.SportScanner might have
1
(Highlights)


If you use Season Folders you don't need them

Posted: 15 Feb 2022 19:06
jbalcorn
Posts: 20
Joined: 2022-01-24

Plex will handle just about any media type.

Posted: 15 Feb 2022 23:45
sport1234
Posts: 20
Joined: 2022-02-08

Hi there, and thank you for the feedback. I still cannot get the sports metadata to work. I moved to a QNAP Plex server, and I have set up a Sports Library for testing purposes as TV Shows, using the Plex Series Scanner as the scanner, and the Sportscanner as the agent. First of all, is this the right configuration for the Sportscanner?

Then as a test, I have the following directory set up:
Sport/English_Premier_League/Season 2122

And the file that I am testing with is:
English.Premier.League.2021-10-22.Arsenal.vs.Aston.Villa (This was the naming convention used in the SPortsdb for this event).

Can someone help to tell me what am I doing Wrong, and the Sportscanner is not finding this file at all.

Thank you in advance.

Posted: 16 Feb 2022 14:45
jbalcorn
Posts: 20
Joined: 2022-01-24

Scanner and Agent should both be SportScanner - it comes with a Series Scanner (SportScanner.py) and a Agent (SportScanner.Bundle)



Posted: 16 Feb 2022 14:48
jbalcorn
Posts: 20
Joined: 2022-01-24

I probably sent you down the wrong path - I have a small edit in my __init__.py where I say

show_title = media.show.replace('_',' ')

So you'll probably want the folder to be

"English Premier League"

with the spaces. Do you have a SportScanner.txt with the contents

xxyy 7,1

to signify how the seasons are split up? Either that or create a folder

English Premier League/Season 2122/

and put the file in there. Then with the SportScanner scanner AND SportScanner agent it should find it.

Posted: 16 Feb 2022 14:49
jbalcorn
Posts: 20
Joined: 2022-01-24

Oh, and you do have your file with the extension, correct? .ts or .mp4 or .mkv or something?

Posted: 16 Feb 2022 21:19
sport1234
Posts: 20
Joined: 2022-02-08

Hi JBalcorn.

Ok. This is what I have.

First, I installed SportScanner correctly on the QNAP NAS, and now I can see the SportScanner in the Scanner and Agent, and now I know that it is correctly set up.

The code I have retrieved it from Github, so it should be correct.

My Directory in (QNAP NAS):
/share/CACHEDEV1_DATA/Multimedia/Sport/English Premier League/Season 2122

And the only file that I have for testing purpose is:
English.Premier.League.2021-10-22.Arsenal.vs.Aston.Villa

But still nothing. Whatever I do, it does not read it. Anything else, I should try, or anything else that you see wrong with this.

I really appreciate your help with this, because I have many sports files, and would like to set them up on Plex.



Posted: 17 Feb 2022 23:51
sport1234
Posts: 20
Joined: 2022-02-08

Hi,

can someone help. I am still trying to get this to work, and from the Plex Media Server Scanner log file, when I search Sports for metadata, I get the following from the Plex Scanner Log file.

Scanning Sports using en(SportScanner) with 0 current media items and 1 section locations in the database.
Performing a scan with 'SportScanner' (language: en virtual: 0).
Scanning /share/CACHEDEV1_DATA/Multimedia/Sport/Soccer/English Premier League/2021-2022
Scanner: Processing directory /share/CACHEDEV1_DATA/Multimedia/Sport/Soccer/English Premier League/2021-2022 (parent: no)
Directory had 1 files, database had 0 files, can't skip.

It seems that it is using the SportScanner correctly, it finds the file in the directory, but it does not match the file or the filename, but I cannot figure out why it does not match? I think I am using the right nomenclature from theSportDb file naming convention?

If someone can help, I would really appreciate it.



Posted: 27 Feb 2022 21:13
jbalcorn
Posts: 20
Joined: 2022-01-24

We'll need more log files because what you posted is actually correct - it sees a file in the directory, the database doesn't have a record of that file, so it should be correct. It's what comes after that we'll need to see.

What you're seeing is the output from the Series Scanner. You could put some print statements into Scanners/Series/SportScanner.py - in the code you'll see a lot commented out, you could uncomment those to see more information in the Scanner log.

But you also want to look at Logs/PMS Plugin Logs/com.plexapp.agents.sportscanner.log to see the output from SportScanner.bundle/Contents/Code/__init__.py which is the Agent scanner.

Also - make sure you have the latest and greatest code. The API key from January was turned off, you'll need the new code with the obfuscated key. If you get it working, consider becoming a dataDB Patreon and getting your own key!

zag
Posted: 01 Mar 2022 21:31
sport1234
Posts: 20
Joined: 2022-02-08

Hi. Thank you for the reply.

I had a router issue, which I just fixed today, abs will need to re build my NAS network, etc.

I will perform your suggestion with the print lines, and show you the output sometime next week, so whenever you have time next week, if you can take a look in this thread, and let me know what you think. I appreciate your help


Posted: 01 Mar 2022 21:33
sport1234
Posts: 20
Joined: 2022-02-08

One more question.

The new code should be on GitHub? Also, I am assuming that if I have the new code, the key is in there?

And yes. Once I get it to work, I had planned on paying and getting my own key.

Thank you again for your help, abs I really appreciate it

Posted: 02 Mar 2022 22:15
jbalcorn
Posts: 20
Joined: 2022-01-24

Yes. We had a new API key issued and obfuscated it in the code so it can't be easily scraped. If you download the latest from github it has a working key (And the code to allow you to use your own)

zag

Who is Online?

In total there are 68 users online :: 3 registered, 0 hidden and 65 guests (based on users active over the past 5 minutes) Most users ever online was 424 on Fri Nov 10, 2017 9:02 pm

About Us

Discussion forum for TheSportsDB.com site and related topics

Rules

- Be Polite
- Respect other users
- Always post log files with issues
- Try to be helpful
- No Piracy discussion

Showing 25 to 50 (Total: 102)