crash when adding users to groups or creating events

[Clerian]

I just upgraded to indico 1.1 using easy_install on Fedora 18, what happened is that whenever I try to change something related to a user, including creating net events the system crashes:

steps to reproduce:

create an event OR modify a group

error:

.... lots of non interesting backtrack steps ...

File "/usr/lib/python2.7/site-packages/indico-1.1-py2.7.egg/MaKaC/user.py", line 130, in addMember

newMember.linkTo(self, "member")

File "/usr/lib/python2.7/site-packages/indico-1.1-py2.7.egg/MaKaC/user.py", line 762, in linkTo

self.linkedTo[field][role].add(obj)

AttributeError?: 'list' object has no attribute 'add'

Solution:

on line 762 in users.py:
change self.linkedTo[field][role].add(obj) to self.linkedTo[field][role].append(obj)

then everything works again.

Best regards.

[Clerian]

Additional information:

Fedora 18
Python 2.7.3
Apache Server version: Apache/2.4.6 (Fedora)
mod_wsgi version 3.4-6.fc18

[Clerian]

Additional error:
when using the "My Profile" button I get the following error message:

Error:

2013-10-03 10:44:37,596 indico.requestHandler: ERROR - Request 140253344110992 failed: "invalid argument"
Traceback (most recent call last):

File "/usr/lib/python2.7/site-packages/indico-1.1-py2.7.egg/MaKaC/webinterface/rh/base.py", line 586, in process

res = self._process()

File "/usr/lib/python2.7/site-packages/indico-1.1-py2.7.egg/MaKaC/webinterface/rh/users.py", line 302, in _process

return p.display()

File "/usr/lib/python2.7/site-packages/indico-1.1-py2.7.egg/MaKaC/webinterface/pages/base.py", line 190, in display

self._display( params ), \

File "/usr/lib/python2.7/site-packages/indico-1.1-py2.7.egg/MaKaC/webinterface/pages/main.py", line 155, in _display

body = WMainBase(self._getBody( params ), self._timezone, self._getNavigationDrawer(),

File "/usr/lib/python2.7/site-packages/indico-1.1-py2.7.egg/MaKaC/webinterface/pages/admins.py", line 1514, in _getBody

html = wcomponents.WTabControl( self._tabCtrl, self._getAW() ).getHTML( self._getTabContent( params ) )

File "/usr/lib/python2.7/site-packages/indico-1.1-py2.7.egg/MaKaC/webinterface/pages/admins.py", line 1554, in _getTabContent

return c.getHTML(params)

File "/usr/lib/python2.7/site-packages/indico-1.1-py2.7.egg/MaKaC/webinterface/wcomponents.py", line 202, in getHTML

vars = self.getVars()

File "/usr/lib/python2.7/site-packages/indico-1.1-py2.7.egg/MaKaC/webinterface/pages/admins.py", line 1422, in getVars

html_varscategories? = user.getRelatedCategories()

File "/usr/lib/python2.7/site-packages/indico-1.1-py2.7.egg/MaKaC/user.py", line 725, in getRelatedCategories

for categ in union(favorites, managed):

TypeError?: invalid argument

Steps to reproduce:

got to My Profile, and there to "Dashboard" or Authorized apps

[arescope]

Dear Clearian,

Did you run the migrate.py script? There is a task called convertLinkedTo that changes the schema of that.

Best regards,

Alberto

[arescope]

Dear Clearian,

Did you manage to fix it?

Best regards

[svalle]

I have the exact same problem as in this ticket. Both the original problem which seems to be fixed by changing add to append, and the 2nd error with

...
for categ in union(favorites, managed):

Actually for me this error is shown already at the link to show 'HOSTNAME/userDashboard.py?userId=0'

Running migrate.py as shown below, does not fix the problem - you still get an error message

python migrate.py --run-only convertLinkedTo

Complete error report.

indico.requestHandler - ERROR base.py:380 -- Request 140398563408720 failed: "invalid argument"
Traceback (most recent call last):
 File "/usr/local/lib/python2.7/dist-packages/indico-1.1.2-py2.7.egg/MaKaC/webinterface/rh/base.py", line 586, in process
   res = self._process()
 File "/usr/local/lib/python2.7/dist-packages/indico-1.1.2-py2.7.egg/MaKaC/webinterface/rh/users.py", line 302, in _process
   return p.display()
 File "/usr/local/lib/python2.7/dist-packages/indico-1.1.2-py2.7.egg/MaKaC/webinterface/pages/base.py", line 190, in display
   self._display( params ), \
 File "/usr/local/lib/python2.7/dist-packages/indico-1.1.2-py2.7.egg/MaKaC/webinterface/pages/main.py", line 155, in _display
   body = WMainBase(self._getBody( params ), self._timezone, self._getNavigationDrawer(),
 File "/usr/local/lib/python2.7/dist-packages/indico-1.1.2-py2.7.egg/MaKaC/webinterface/pages/admins.py", line 1514, in _getBody
   html = wcomponents.WTabControl( self._tabCtrl, self._getAW() ).getHTML( self._getTabContent( params ) )
 File "/usr/local/lib/python2.7/dist-packages/indico-1.1.2-py2.7.egg/MaKaC/webinterface/pages/admins.py", line 1554, in _getTabContent
   return c.getHTML(params)
 File "/usr/local/lib/python2.7/dist-packages/indico-1.1.2-py2.7.egg/MaKaC/webinterface/wcomponents.py", line 190, in getHTML
   vars = self.getVars()
 File "/usr/local/lib/python2.7/dist-packages/indico-1.1.2-py2.7.egg/MaKaC/webinterface/pages/admins.py", line 1422, in getVars
   html_vars["categories"] = user.getRelatedCategories()
 File "/usr/local/lib/python2.7/dist-packages/indico-1.1.2-py2.7.egg/MaKaC/user.py", line 725, in getRelatedCategories
   for categ in union(favorites, managed):
TypeError: invalid argument

Could it be some python library indico depends on which is a too new/old version?
Indico was installed using easy_install.

[arescope]

Which version of ZODB do you have installed?

Best regards,

Alberto

[svalle]

I have the most recent version suggested by easy_install

ls -d /usr/local/lib/python*/dist-packages/ZODB*
/usr/local/lib/python2.7/dist-packages/ZODB
/usr/local/lib/python2.7/dist-packages/ZODB3-3.10.5-py2.7-linux-x86_64.egg
/usr/local/lib/python2.7/dist-packages/ZODB3-3.11.0a3.egg-info
/usr/local/lib/python2.7/dist-packages/ZODB-4.0.0.egg-info

importing ZODB from inside python installs /usr/local/lib/python2.7/dist-packages/ZODB, which as far as I can see is version 4?

[jbenito]

Precisely, that's the issue. Indico does not support ZODB4. We can ensure everything works fine with ZODB - 3.10.5 as recommended here:

​https://github.com/indico/indico/blob/v1.1/requirements.txt#L6

You will need to remove all references to ZODB-4. If Indico still fails to run, I would remove everything ZODB-related form inside dist-packages and do:

$ pip install -r requirements.txt

[svalle]

I've deleted all python modules related to Indico including ZODB.
Indico was reinstalled + everything else was reinstalled using the suggested pip command above. Now I have the following ZODB packages:

ls -d /usr/local/lib/python*/dist-packages/ZODB*
/usr/local/lib/python2.7/dist-packages/ZODB
/usr/local/lib/python2.7/dist-packages/ZODB3-3.10.5-py2.7.egg-info

i.e., only the required ZODB version. Unfortunately I still get the same TypeError: invalid argument error as above.

While testing this I can see that I can only trigger the error if I use an old user (created before the upgrade to Indico 1.1), i.e., maybe this is due to an inconsistent database...

[svalle]

Hmmm... The fix in the original ticket only works sometimes. The diff below seems to work. Without it I get either
AttributeError: 'list' object has no attribute 'add'
or
AttributeError: 'BTrees.OOBTree.OOTreeSet' object has no attribute 'append'

Diff that works (But the problem is most probably elsewhere...)

<                 self.linkedTo[field][role].add(obj)
---
>                 if type(self.linkedTo[field][role]) == list:
>                     self.linkedTo[field][role].append(obj)
>                 else:
>                     self.linkedTo[field][role].add(obj)

Complete backtrace

indico.requestHandler - ERROR base.py:380 -- Request 140619150233936 failed: "'BTrees.OOBTree.OOTreeSet' object has no attribute 'append'"
Traceback (most recent call last):
 File "/usr/local/lib/python2.7/dist-packages/indico-1.1.2-py2.7.egg/MaKaC/webinterface/rh/base.py", line 586, in process
   res = self._process()
 File "/usr/local/lib/python2.7/dist-packages/indico-1.1.2-py2.7.egg/MaKaC/webinterface/rh/registrationFormDisplay.py", line 181, in _process
   rp.setAvatar(user)
 File "/usr/local/lib/python2.7/dist-packages/indico-1.1.2-py2.7.egg/MaKaC/registration.py", line 4782, in setAvatar
   a.linkTo(self, "registrant")
 File "/usr/local/lib/python2.7/dist-packages/indico-1.1.2-py2.7.egg/MaKaC/user.py", line 765, in linkTo
   self.linkedTo[field][role].append(obj)
AttributeError: 'BTrees.OOBTree.OOTreeSet' object has no attribute 'append'

[jmonnich]

Did you run the migration script? The convertLinkedTo migration step converts all those lists to OOTreeSets. You can safely run that migration task again by the way to ensure all those entries have been converted.

[svalle]

Hi jmonnich,

I previously ran migrate.py as shown in comment comment:9, but apparently convertLinkedTo only runs if you specify that the previous version was 1.0... I now get the error below:

Would it be safe to change the line to if True? Comparing with the lines below, the if expression seems only to be an optimisation as to not check all links?

python migrate.py --run-only convertLinkedTo --prev-version 1.0

This script will migrate your Indico DB to a new version. We recommend that
this operation be executed while the web server is down, in order to avoid
concurrency problems and DB conflicts.


Are you sure you want to execute the migration now? [y/N] y

Executing migration...

Probing DB connection... DONE!

# Convert Avatar.linkedTo structure to use OOTreeSets and import linkedTo information into Redis (if enabled) (1.1)
Note: Some links might point to broken objects which will be skipped automatically.

Migration failed! DB may be in  an inconsistent state:
Traceback (most recent call last):
  File "migrate.py", line 897, in main
    dry_run=args.dry_run)
  File "migrate.py", line 843, in runMigration
    task(dbi, withRBDB, prevVersion)
  File "migrate.py", line 762, in convertLinkedTo
    if hasattr(linkedTo[field][role][0], 'getConference'):
TypeError: 'BTrees.OOBTree.OOTreeSet' object does not support indexing

[jmonnich]

Sorry for the delay, looks like I missed the notification email for your comment...

Please try replacing this line:

if hasattr(linkedTo[field][role][0], 'getConference'):

with this:

if isinstance(linkedTo[field][role], list) and hasattr(linkedTo[field][role][0], 'getConference'):

[svalle]

Thanks for looking into this again.
I actually fixed this using if True, and then adding a check further down for whether each obj had a getConference method.
Everything seems to work now.

[billm]

Hi
We're also getting the same errors as described above, this on a system using 1.1.2. The errors are the same as described above, but unfortunately the fixes get us back to what Comment 14 shows. Yes, we've tried running the maigrate script, and even modified the user.py file.

svalle, can you send/post a patch of what you did to rectify the problem? We're dead in the water and hate to have to reinstall everything including hand-enter many meetings back in again.

Please and thanks.

[billm]

Replying to svalle:

Thanks for looking into this again.
I actually fixed this using if True, and then adding a check further down for whether each obj had a getConference method.
Everything seems to work now.

Can you post a patch please?

[svalle]

Hi billm

See the diff below.
In our case, the script never went into the fixme_fixme case (where migration immediately stops).
We need some with more understanding of the indico db to create a prober patch, if they want.

Good luck!

--- migrate.py	2014-01-31 19:06:16.000000000 +0100
+++ migrate.svalle.py	2014-03-13 10:05:54.463748311 +0100
@@ -759,13 +759,14 @@
                 todo = set(linkedTo[field][role])
                 # We have broken objects in the database which will fail in the getConference() call. If the current
                 # object type has such a method call it on each object and skip it in case it raises an AttributeError
-                if hasattr(linkedTo[field][role][0], 'getConference'):
+                if True:
                     for obj in linkedTo[field][role]:
                         try:
                             obj.getConference()
                         except AttributeError, e:
                             print '  \tSkipping broken object in %s/%s/%s: %r' % (avatar.getId(), field, role, obj)
                             todo.remove(obj)
+                            fixme_fixme # we do not actually want this to run! - maybe we are removing something we don't want (see ticket)
                 avatar.linkedTo[field][role].update(todo)
         if use_redis:
             avatar_links.init_links(avatar, client=pipe)