[Tikiwiki-devel] tiki cluster?

Post by James H. Thompson
Anyone running Tiki on multiple machines sharing a common database?
Does Tiki work in this kind of configuration?
I'm considering Tiki for an application that would need to support
several hundred simultaneous users.
Thanks
Jim
James H. Thompson

I doubt that anyone has done this before. While it should work in
theory, my suggestion is to get a well-endowed server and try it as a
single server. It should be simple to split the DB and middle tier if
you want, but the network overhead for db accesses may negate the memory
/cpu savings. I'd done a lot of cluster work with Oracle databases, but
not for supporting large numbers of small requests, rather it was to
support very large queries or to provide multi-site redundancy.

I believe that a top-of-the line SMP server with several GB of ram and a
fast SCSI subsystem running a modern version of Linux will handle your
load with ease. The key is to reduce all the bottlenecks you can on the
system. This sort of system might easily saturate your network links, so
don't forget the NIC's.

Dennis

Ross Smith II

2003-08-01 13:38:54 UTC

Post by Dennis Heltzel
I believe that a top-of-the line SMP server with several GB of ram and a
fast SCSI subsystem running a modern version of Linux will handle your
load with ease. The key is to reduce all the bottlenecks you can on the
system. This sort of system might easily saturate your network links, so
don't forget the NIC's.

Dennis,

Honestly, as much as I love tiki, I don't see it winning any speed contests. It usually takes >1 second to image a page. To
support 100+ users, we need to reduce that considerably. Throwing big iron at the problem will only help so much.

Luis was of a mind that 1.8 would be a "no new features" release, but would include abstracting the db, cleaning up the code, and
improving performance.

I suggest we have a goal of improving performance by a factor of 10.

Whadda say?

-Ross

Luis Argerich

2003-08-01 13:50:19 UTC

As I've said many times I'm against a generic initiative to optimize or
increase performance. It will only make the
code hard to understand and the problems would eventually outweight the
performance increase.
I do want to improve the performance but in order to to it we must find the
20% of Tiki that is taking the 80% of the
time that is needed to render a page. Once a good profiling job is done and
the bottleneck is found we can concentrate
efforts to improve performance in that specific area without caring about
other problems. This can be repeated as many
times as needed.

----- Original Message -----
From: "Ross Smith II" <***@smithii.com>
To: <***@adolor.com>
Cc: <tikiwiki-***@lists.sourceforge.net>
Sent: Friday, August 01, 2003 10:38 AM
Subject: RE: [Tikiwiki-devel] Re: tiki cluster?

Dennis,
Honestly, as much as I love tiki, I don't see it winning any speed

contests. It usually takes >1 second to image a page. To

Post by Ross Smith II
support 100+ users, we need to reduce that considerably. Throwing big

iron at the problem will only help so much.

Post by Ross Smith II
Luis was of a mind that 1.8 would be a "no new features" release, but

would include abstracting the db, cleaning up the code, and

Post by Ross Smith II
improving performance.
I suggest we have a goal of improving performance by a factor of 10.
Whadda say?
-Ross
-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including
Data Reports, E-commerce, Portals, and Forums are available now.
Download today and enter to win an XBOX or Visual Studio .NET.

http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01

Post by Ross Smith II
_______________________________________________
Tikiwiki-devel mailing list
https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.505 / Virus Database: 302 - Release Date: 7/30/2003

Jeremy Jongsma

2003-08-01 16:09:46 UTC

Tracking down and eliminating the performance bottlenecks? That sounds
like a generic initiative to increase performance to me... ;)

I've been hopping from one open source CMS to another, for one reason: I
want to run a business on one. But so far, they have all been so
completely focused on features that they forget that what most of the
world really needs is scalability.

I don't think we really need to add any more features for 1.8 (or at
least it shouldn't be the primary focus) - feature for feature, we've
got better modules than anyone. But it seems we're still lacking in the
same place everyone else is - scalability.

The next step for Tiki should be making sure it's capable of powering a
moderately high traffic web site with the minimum hardware possible.
Otherwise, it will only be adopted by small sites and hobbyists. We've
got too many good developers here to let that happen.

-j

Post by Luis Argerich
As I've said many times I'm against a generic initiative to optimize or
increase performance. It will only make the
code hard to understand and the problems would eventually outweight the
performance increase.
I do want to improve the performance but in order to to it we must find the
20% of Tiki that is taking the 80% of the
time that is needed to render a page. Once a good profiling job is done and
the bottleneck is found we can concentrate
efforts to improve performance in that specific area without caring about
other problems. This can be repeated as many
times as needed.
----- Original Message -----
Sent: Friday, August 01, 2003 10:38 AM
Subject: RE: [Tikiwiki-devel] Re: tiki cluster?

Dennis,
Honestly, as much as I love tiki, I don't see it winning any speed

contests. It usually takes >1 second to image a page. To

Post by Ross Smith II
support 100+ users, we need to reduce that considerably. Throwing big

iron at the problem will only help so much.

Post by Ross Smith II
Luis was of a mind that 1.8 would be a "no new features" release, but

would include abstracting the db, cleaning up the code, and

http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01

Post by Ross Smith II
_______________________________________________
Tikiwiki-devel mailing list
https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.505 / Virus Database: 302 - Release Date: 7/30/2003
-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including
Data Reports, E-commerce, Portals, and Forums are available now.
Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01
_______________________________________________
Tikiwiki-devel mailing list
https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel

--
Jeremy Jongsma
Co-Founder
TickChat
***@tickchat.com
http://www.tickchat.com

Michael S. Zick

2003-08-01 16:54:11 UTC

Jeremy Jongsma

2003-08-01 18:42:37 UTC

I agree with Michael.

Also, for people who recently starting working on Tiki (like me), the
code isn't necessarily all that easy to understand in its current state.
The rush to add features and patch bugs seems to have resulted in some
bloated pages with a logical flow that is difficult to quickly follow.
I think that overall, a generic code cleanup / optimization initiative
would end up making the code more readable, scalable and maintainable.

To further this objective and prepare Tiki for the world stage, I
propose creating an "Architecture" team, composed of our best and
brightest. This group would be responsible for:

1) Refactoring Tiki as necessary to make it modular and extendable
(i.e. separating core from modules)
2) Proposing (and enforcing?) a Tiki API (particularly for module dev)
3) Ensuring acceptable performance of the system as a whole
4) Addressing scalability issues for multi-server environments

I think the team members should definitely include Luis, Mose, Ross, and
Flo. Because of the impact this team would have on the foundation of
Tiki, I'd say any additional members would be by invitation from them only.

Imagine the day when we can convince companies that Tiki is a reliable,
scalable and fast solution for all their web site needs. I'm fully
confident that we can get there, with the focused guidance of this team.

And then we can all quit our day jobs and become Tiki consultants. :)

-j

Post by Michael S. Zick

I have to disagree.
There can be many places to both improve the code quality, including
maintainability, while improving performance.
A lot of the "just-thrown-together" and "it-works-for-me" code could
easily be revised to meet both goals.
Mike
-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including
Data Reports, E-commerce, Portals, and Forums are available now.
Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01
_______________________________________________
Tikiwiki-devel mailing list
https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel

--
Jeremy Jongsma
Co-Founder
TickChat
***@tickchat.com
http://www.tickchat.com

Dennis Heltzel

2003-08-01 19:07:06 UTC

Post by Jeremy Jongsma
1) Refactoring Tiki as necessary to make it modular and extendable
(i.e. separating core from modules)
2) Proposing (and enforcing?) a Tiki API (particularly for module dev)
3) Ensuring acceptable performance of the system as a whole
4) Addressing scalability issues for multi-server environments
I think the team members should definitely include Luis, Mose, Ross, and
Flo. Because of the impact this team would have on the foundation of
Tiki, I'd say any additional members would be by invitation from them only.

I respectfully disagree. I don't think it's likely that any sort of
"feature freeze" will be enforced because careful enforcement is not the
culture that tiki is built on. I really doubt that the market you are
targetting will be our if we just follow those steps anyway.

I'll leave it up to the people you mentioned to weigh in with their
opinions, but I expect a lukewarm reaction, at best.

If there are a number of devs who wish to do this, I don't see any
problem with them starting a new CVS branch to make the changes they
want. they can build and test in parallel with the main trunk and when
they have tiki re-written to be a better tiki, then we can all switch
over. This would essentially be a code fork until the massive changes
you are suggesting are complete. If the devs on that branch slow down or
give up, we haven't lost anything, if they succeed, we gain much without
the "growing pains" involved with co-opting the entire codebase.

Luis Argerich

2003-08-01 19:17:07 UTC

I agree with Dennis.

----- Original Message -----
From: "Dennis Heltzel" <***@adolor.com>
To: <tikiwiki-***@lists.sourceforge.net>
Sent: Friday, August 01, 2003 4:07 PM
Subject: [Tikiwiki-devel] Re: TikiTeamArchitecture

Post by Dennis Heltzel

I respectfully disagree. I don't think it's likely that any sort of
"feature freeze" will be enforced because careful enforcement is not the
culture that tiki is built on. I really doubt that the market you are
targetting will be our if we just follow those steps anyway.
I'll leave it up to the people you mentioned to weigh in with their
opinions, but I expect a lukewarm reaction, at best.
If there are a number of devs who wish to do this, I don't see any
problem with them starting a new CVS branch to make the changes they
want. they can build and test in parallel with the main trunk and when
they have tiki re-written to be a better tiki, then we can all switch
over. This would essentially be a code fork until the massive changes
you are suggesting are complete. If the devs on that branch slow down or
give up, we haven't lost anything, if they succeed, we gain much without
the "growing pains" involved with co-opting the entire codebase.
-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including
Data Reports, E-commerce, Portals, and Forums are available now.
Download today and enter to win an XBOX or Visual Studio .NET.

http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01

Post by Dennis Heltzel
_______________________________________________
Tikiwiki-devel mailing list
https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.505 / Virus Database: 302 - Release Date: 7/30/2003

Ben Tallman

2003-08-06 20:44:06 UTC

Did we ever reach any sort of consensus on this? IMNSHO the amount of
discussion warrants a team, TikiTeamArchitecture, whose job it will be
to determine its own requirements, as the stearing body...

Ben

Ben Tallman

2003-08-01 21:09:01 UTC

In my mind the real issue is this: Is tiki an application, or a
framework?

I am using tiki as an applicaiton framework.
1) It supports various forms of user authentication/permissions and
their management
2) It has configurable menus, etc...
3) Fairly well documented for the end user
4) Runs almost anywhere.
5) GREAT dev community

As an added benefit for "intranet in a box" it:
1) Has article/FAQ/wiki/user management
2) Supports LDAP
3) Webmail

It does not however offer well documented API's. In fact it isn't
really that simple to figure out where a particular function will end up
being in the code-base.

I also use tiki as an application:
1) Wiki
2) CMS (articles)
3) Tasks
4) User Management
5) Extensible
6) GREAT dev community

I believe that either way, standardizing a few things can MAJORLY
improve the "developability" of the app or framework. This isn't as big
a task as you might imagine. At least I don't think so anyway ( I am
famous for my impossible deadlines though).

Luis has mentioned that he believes the code is more easily readable in
it's current form that if it were optimized? Not sure how that can be a
conclusion. Tiki is currently a very complex app, but given a few days
of digging, you can get your hands around it. I think that with some
fairly basic structural changes, it would be possible to get your hands
around the app MUCH quicker, and at the same time, performace could be
vastly improved. Basically, my proposal would be that the directory
structure would change. I think that ALL code needed to run a feature
should be in a single directory (or tree) based in /features/feature_name/.

I have been pondering this for about a week now, and am about ready to
build a wiki page on the subject. To begin with I created
ModuleCreationDev and FeatureCreationDev to make sure that I understood
the existing system. I would like more feedback (none to date) just to
make sure that I haven't missed something big.

IMHO I don't think that this lends itself well to a 1.8 version at all,
but instead to a 2.0 version. This will be a LARGE project, but not one
that is undo-able at all. I was amazed at the level of commitment
dosplayed during the 1.7 release process, and therefore I think that the
major work isn't the development of the structure, but instead the
refactoring of the existing features. Once the new base API is working,
the features could get moved over time. I am assuming that this could
be done WHILE 1.8 is being worked on...

Ben

PS - I think TikiTeamArchitecture is a requirement.

Michael S. Zick

2003-08-01 22:11:08 UTC

On Friday 01 August 2003 04:09 pm, Ben Tallman wrote:
- - - - big snip - - - -

Post by Ben Tallman
Luis has mentioned that he believes the code is more easily readable in
it's current form that if it were optimized? Not sure how that can be a
conclusion. Tiki is currently a very complex app, but given a few days
of digging, you can get your hands around it. I think that with some
fairly basic structural changes, it would be possible to get your hands
around the app MUCH quicker, and at the same time, performace could be
vastly improved. Basically, my proposal would be that the directory
structure would change. I think that ALL code needed to run a feature
should be in a single directory (or tree) based in /features/feature_name/.

I proposed something similar in principle, can't find the ref. so I'll
summarize:
The basic principle divides the display of a page into two parts:
1) ...something/somepart.PHP The logic.
2) ...something/somepart.TPL The display layout.
To which, Tiki adds the use of Smarty.
The current practice is to put NEARLY ALL of the smarty->assign
statements in tiki-setup.php. Which executes all of them for every
page request.
I proposed that the structure be extended to a third part:
3) ...something/somepart.INC The smarty->assign for (1) & (2)
Gains:
a) Developer of a page/feature knows what variables are referred
too in the *.TPL and *.PHP - now they have one place for the
smarty->assigns.
b) Maintainer has one place to look/maintain the assignments.
c) tiki-setup.php now becomes a set of conditional includes...
That is; each page request only executes the smarty->assigns
that are required for that page.

Which is similar to your proposal, and I think provides an
example of performance improvement AND code improvement.

Mike

Gustavo Muslera

2003-08-02 02:05:14 UTC

Post by Jeremy Jongsma
Also, for people who recently starting working on Tiki (like me), the
code isn't necessarily all that easy to understand in its current state.
The rush to add features and patch bugs seems to have resulted in some
bloated pages with a logical flow that is difficult to quickly follow.
I think that overall, a generic code cleanup / optimization initiative
would end up making the code more readable, scalable and maintainable.
To further this objective and prepare Tiki for the world stage, I
propose creating an "Architecture" team, composed of our best and
1) Refactoring Tiki as necessary to make it modular and extendable
(i.e. separating core from modules)
2) Proposing (and enforcing?) a Tiki API (particularly for module dev)
3) Ensuring acceptable performance of the system as a whole
4) Addressing scalability issues for multi-server environments

Jeremy Jongsma

2003-08-02 18:06:21 UTC

Is anyone opposed to trying this incrementally? Starting with Michael's
suggestions would be a great start to make Tiki more modular and efficient.

Our team is growing fast, and I think it would be in everybody's best
interest to make these changes in order to allow the concept of "feature
teams" that, if desired, can work independently of others without
worrying too much about what other sections they're affecting, or if
another team is modifying the same files.

Not to mention that if we're going to be able to convince third parties
to develop their modules for Tiki, we need a well-defined feature API.
That's where an Architecture team would come in. Not to be a "ruling
cabal" of Tiki - but because if we let 60+ people add to the API as they
see fit, we end up like PostNuke - a bloated mess.

-j

I have been writing about this kind of themes here and in tikiwiki.org
(i.e. http://tikiwiki.org/Desintegration) for some time now. But a heavy
reorganization of all modules could be equivalent from start another
(but looking similar) project from zero, in fact, will be easier to
start another project than modify all and each one of the modules for
that future architecture.
But things can be made at a slow motion, defining an architecture,
making it to live along with current architecture, have a recomendation
on how to code for this (but not stoping anyone to work in old-style
modules) and at its own rate, evolve current features to the new
architecture. If the new architecture for integrating external modules
in the core is good enough, then immediately after the release could be
developed outside the main branch and normal Tiki distribution new
modules, while still works all old style development. If later actual
modules are migrated to the new architecture, we can shrink the Tiki
core putting outside non essential modules.
-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including
Data Reports, E-commerce, Portals, and Forums are available now.
Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01
_______________________________________________
Tikiwiki-devel mailing list
https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel

--
Jeremy Jongsma
Co-Founder
TickChat
***@tickchat.com
http://www.tickchat.com

Dave Sanders

2003-08-03 03:56:28 UTC

<snip>
But a heavy
reorganization of all modules could be equivalent from start another
(but looking similar) project from zero, in fact, will be easier to
start another project than modify all and each one of the modules for
that future architecture.

I'm not neccessarily sure thats a bad thing...

Look at it this way: You've taken a bunch of ideas, built them up,
tweaked them, understood what you really needed, how it should work,
etc. Along the way, 60+ developer friends jumped in and had 1000 ideas
of their own, put them in, turned them around, and worked them over.

While Tiki is a well featured product, it does have some maturing to
do. Would it be a "bad thing" to do this for v2.0, by consolidating and
rewriting the base architecture from scratch again - applying all of the
knowledge learned over the past months? While it may seem like a very
daunting task - it shows a certain level of maturity in the project to
be able to rebuild the core architecture toward the ideas that were
actually used and expanded on.

Not pushing any decision here - just playing devil's advocate. :)

D

Michael S. Zick

2003-08-03 14:24:31 UTC

Post by Dave Sanders

I'm not neccessarily sure thats a bad thing...
Look at it this way: You've taken a bunch of ideas, built them up,
tweaked them, understood what you really needed, how it should work,
etc. Along the way, 60+ developer friends jumped in and had 1000 ideas
of their own, put them in, turned them around, and worked them over.
While Tiki is a well featured product, it does have some maturing to
do. Would it be a "bad thing" to do this for v2.0, by consolidating and
rewriting the base architecture from scratch again - applying all of the
knowledge learned over the past months? While it may seem like a very
daunting task - it shows a certain level of maturity in the project to
be able to rebuild the core architecture toward the ideas that were
actually used and expanded on.

Ag reeded.
Restated in other terms;
with some questions key to the process:
Consider these stages of application development in a WEB environment:
1) "Proof of Concept" code (Code adequate to "try-out" an idea)
2) "Prototype" code (Represents the refinement of the details of the idea)
3) "Production" code (Represents the clean integration of the new idea)

Excuse the length to follow, I want to get the reader's thinking in-sync
with the background to the questions.

Tiki-Wiki features and services are each in their own stage, mostly stages
one (1) and two (2).
The stage (1) stuff is just there to "see if idea_x would work and provide
something useful to the whole". It may not (probably does not) work for
the full range of software and systems that the application does.

With "idea_x" doing its original thing (other than crashing the system);
that idea gets refined by deciding what modifications it needs to be an
effective part of the application. Note: The phrase: "effective part" was
not part of the stage 1 specs.

Based on the above, a "stage two (2)" implementation of the code for
"idea_x" is included in Tiki-Wiki.

Whether implicitly by group consensus or explicitly by project policy;
the above is happening relatively smoothly.

The need to promote sections of the Tiki-Wiki code to stage three (3)
is recognized (at least implicitly) by recent posts, including this thread.
Work is proceeding on some of those sections - with signs that there
is more to come as the sections needing to be stage three code are
recognized.

Which brings us to a question key to the future of Tiki-Wiki:
"What is our application?"

Variations of this question have been appearing, both implicitly and
explicitly, in recent posts but it is my own opinion that the question
needs to be addressed as its own subject.
It is the answer to this question that will guide what portions of Tiki-Wiki
need the additional work to promote the code to stage 3 and what can
remain stage 2 code.

The current state of Tiki-Wiki is that of a vast collection of answers to
interactive web site usage - without any hint as to what the question is.
A certain sign that the question "What is Tiki-Wiki?" has not been addressed.

Mike

Michael S. Zick

2003-08-01 16:54:11 UTC

James H. Thompson

2003-08-01 22:42:36 UTC

I think we all share the same goal -- make Tiki faster.
I agree that the first step is to add performance profiling to Tiki.

Often the performance bottlenecks are not where you expect them. The last time I did some very
basic Tiki peformance profiling, the main bottleneck appeared to be the time it took to load the PHP
code -- not the database lookups. This was a while back, so hopefully is no longer the case.

Performance profiling shouldn't be regarded as a one-time exercise. Its on-going and the tools need
to built into the product.

Maybe the first goal should be for Tiki to have the best built-in peformance profiling for any CMS
or large PHP application.

After that the performance issues would be easily addressed, and we could discuss how to fix them
rather than where they might be.

Jim

James H. Thompson
***@lava.net

----- Original Message -----
From: "Luis Argerich" <***@fuegolabs.com>
To: <tikiwiki-***@lists.sourceforge.net>; <***@adolor.com>
Cc: <tikiwiki-***@lists.sourceforge.net>
Sent: Friday, August 01, 2003 3:50 AM
Subject: Re: [Tikiwiki-devel] Re: tiki cluster?

Dennis,
Honestly, as much as I love tiki, I don't see it winning any speed

contests. It usually takes >1 second to image a page. To

Post by Ross Smith II
support 100+ users, we need to reduce that considerably. Throwing big

iron at the problem will only help so much.

Post by Ross Smith II
Luis was of a mind that 1.8 would be a "no new features" release, but

would include abstracting the db, cleaning up the code, and

http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01

Post by Ross Smith II
_______________________________________________
Tikiwiki-devel mailing list
https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel

James H. Thompson

2003-08-01 22:42:36 UTC

Dennis,
Honestly, as much as I love tiki, I don't see it winning any speed

contests. It usually takes >1 second to image a page. To

Post by Ross Smith II
support 100+ users, we need to reduce that considerably. Throwing big

iron at the problem will only help so much.

Post by Ross Smith II
Luis was of a mind that 1.8 would be a "no new features" release, but

would include abstracting the db, cleaning up the code, and

http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01

Post by Ross Smith II
_______________________________________________
Tikiwiki-devel mailing list
https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel

Jeremy Jongsma

2003-08-01 16:19:05 UTC

+1

-j

Post by Ross Smith II
Luis was of a mind that 1.8 would be a "no new features" release, but would include abstracting the db, cleaning up the code, and
improving performance.
I suggest we have a goal of improving performance by a factor of 10.
Whadda say?
-Ross
-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including
Data Reports, E-commerce, Portals, and Forums are available now.
Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01
_______________________________________________
Tikiwiki-devel mailing list
https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel

--
Jeremy Jongsma
Co-Founder
TickChat
***@tickchat.com
http://www.tickchat.com

Gustavo Muslera

2003-08-01 14:52:16 UTC

As most things reside in a mysql database, maybe it could work having
multiple web servers accessing the same database server, and there
accessing from several computers at the same time should work.

But there are some other things that resides in the hard disk also, some
of them shareable (i.e. that can be solved with a a common nfs
directory, for things like file galleries stored on filesystem instead
of database), and some maybe not (templates_c?), and some gray area in
the middle (writable directories in wiki installation in general, not
entirely sure how much they should work in such environment).

Ross Smith II

2003-08-01 14:56:28 UTC

Luis,

I agree 100%.

ezPublish has an excellent feature that allows you to turn on debugging and see exactly how long each step takes in imaging a page.

My guess is that our queries are where tiki needs to most help.

First, we could add query timings and EXPLAIN each query. My guess is we're doing table scans when we should be using indexes. This
is really simple to do, but will yield huge benefits.

Second, we could remove all unneccessay SELECT COUNT(*)'s, which are very expensive DB wise, as they do a full index scan. For
example:

$cant = $this->getOne("select count(*) from tiki_referer_stats where referer='$referer'");
if($cant) {
$query = "update tiki_referer_stats set hits=hits+1,last=$now where referer='$referer'";
} else {
$query = "insert into tiki_referer_stats(referer,hits,last) values('$referer',1,$now)";
}

would be better written as:

$query = 'update tiki_referer_stats set hits = hits + 1, last = ? where referer = ?';
$tikilib->query($query, array($now, $referer));
if ($tikilib->db->affectedRows() == 0) {
$query = 'insert into tiki_referer_stats (referer, last, hits) values (?, ?, ?)';
$tikilib->query($query, array($referer, $now, 1));
}

This not only gets rid of the SELECT COUNT(*), but replaces two queries with one, in most cases.

We could even generalize this as:

$fields = array(
'referer' => $referer,
'last' => $now,
'hits' => array('insert' => 1, 'update' => '~hits + 1'),
);
$tikilib->replace('tiki_referer_stats', $fields, 'referer');

Which is a lot easier to read than the first two examples, IMHO.

Lastly, we could cache preferences, and other information that changes infrequently, in $_SESSION, and update it only if the
information changes (such as the user changing their preferences).

-Ross

Ben Tallman

2003-08-01 17:12:48 UTC

IMHO -

The issue is probably NOT db related as much as it is that we reload ALL
permissions and setup for every page. I think it would be better to
call a function to verify permissions, OR store them in the session.
Grab them when needed. This could be a HUGE improvement in speed.
Single table, single column queries are VERY fast, and also the only
thing that a developer is really allowed to optimize in tiki, and
therefore, not likely to be the root of the problem.

Ben

Luis,
I agree 100%.
ezPublish has an excellent feature that allows you to turn on debugging and see exactly how long each step takes in imaging a page.
My guess is that our queries are where tiki needs to most help.
First, we could add query timings and EXPLAIN each query. My guess is we're doing table scans when we should be using indexes. This
is really simple to do, but will yield huge benefits.
Second, we could remove all unneccessay SELECT COUNT(*)'s, which are very expensive DB wise, as they do a full index scan. For
$cant = $this->getOne("select count(*) from tiki_referer_stats where referer='$referer'");
if($cant) {
$query = "update tiki_referer_stats set hits=hits+1,last=$now where referer='$referer'";
} else {
$query = "insert into tiki_referer_stats(referer,hits,last) values('$referer',1,$now)";
}
$query = 'update tiki_referer_stats set hits = hits + 1, last = ? where referer = ?';
$tikilib->query($query, array($now, $referer));
if ($tikilib->db->affectedRows() == 0) {
$query = 'insert into tiki_referer_stats (referer, last, hits) values (?, ?, ?)';
$tikilib->query($query, array($referer, $now, 1));
}
This not only gets rid of the SELECT COUNT(*), but replaces two queries with one, in most cases.
$fields = array(
'referer' => $referer,
'last' => $now,
'hits' => array('insert' => 1, 'update' => '~hits + 1'),
);
$tikilib->replace('tiki_referer_stats', $fields, 'referer');
Which is a lot easier to read than the first two examples, IMHO.
Lastly, we could cache preferences, and other information that changes infrequently, in $_SESSION, and update it only if the
information changes (such as the user changing their preferences).
-Ross
-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including
Data Reports, E-commerce, Portals, and Forums are available now.
Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01
_______________________________________________
Tikiwiki-devel mailing list
https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel

Flo G.

2003-08-01 22:27:26 UTC

Post by Dennis Heltzel
I respectfully disagree. I don't think it's likely that any sort of
"feature freeze" will be enforced because careful enforcement is not the
culture that tiki is built on. I really doubt that the market you are
targetting will be our if we just follow those steps anyway.

That's true. with > 60 developers, a feature freeze is hardly possible.

Post by Dennis Heltzel
I'll leave it up to the people you mentioned to weigh in with their
opinions, but I expect a lukewarm reaction, at best.
If there are a number of devs who wish to do this, I don't see any
problem with them starting a new CVS branch to make the changes they
want. they can build and test in parallel with the main trunk and when
they have tiki re-written to be a better tiki, then we can all switch
over. This would essentially be a code fork until the massive changes
you are suggesting are complete. If the devs on that branch slow down or
give up, we haven't lost anything, if they succeed, we gain much without
the "growing pains" involved with co-opting the entire codebase.

That's half true. Massive changes like DB abstraction have to be completed
almost full before these changes can be committed to the main branch.
Otherwise we would make it very hard for other developers.
But there are many things we can do "online". The things i think we have
to inspect/code:
- tiki init (tiki-setup.php, tiki-setup_base.php, setup.php)
- factorisation - if still possible
- a module api

The module api should provide all interfaces to add new modules (or should
i say features) while some/most of the features currently in tiki should
stay there as core components.

Another thing luis mentioned in the "tiki cluster?" thread: Search the 20%
of code that take 80% of time. Example: I've created 5500 dummy tiki pages
(1-64 kB per page) on my performance test system. I couldn't load the
tiki-admin_categories.php. Timeout after 900 sec. Then i saw:
tiki-admin_categories.php provides a dropdown list of wiki pages. To get
this list, it used a query that gave back: the page name, the number of
hits, the number of links .... I added a function that only returns the
page names and now i can load the page in 50 secs. Not fast but it works
now.

Flo

Marc Laporte

2003-08-05 09:25:50 UTC

I also think we should have such a team.

Here is what I had written on http://tikiwiki.org/TikiTeamsPlan

"Development steering team
This dev group must be lead by Luis and other experienced high level
developers. They make sure the overall project is going in right direction
and is future proof -> Master RoadMap.
This team is responsible for major development decisions & strategies. For
example: What is our Release/Branching strategy? Coding standards/style and
developer docs. API docs. Architecture, etc
This team can also be involved in transversal features which affect several
other features.
-Releases, feature freezes
-Responsible to oversee the other dev feature teams.
Decides if a feature is considered official & supported or a 'use at your
own risk' plugin/patch/feature."

I don't have a preference for the team name. Core devs, Team Architecture,
dev steering, etc but I feel we would benefit in the short and long term
from guidance and direction from this team. New devs need a sense of
direction.

Already, many things have been done in this sense. An agreement has been
reached on CVS tagging. A DB abstraction model has been presented. We want
everyone to follow this no? And now, there is a call for a coding standard.

Post by Jeremy Jongsma
1) Refactoring Tiki as necessary to make it modular and extendable
(i.e. separating core from modules)

Refactoring has been done in the last few versions and Tiki is much faster
and requires less RAM

Post by Jeremy Jongsma
3) Ensuring acceptable performance of the system as a whole

Done in general now. Let's formalize that someone is actively testing,
benchmarking and making sure the biggest bottlenecks are eliminated and
update the dev documentation to warn devs not to repeat the same errors.

Post by Jeremy Jongsma
4) Addressing scalability issues for multi-server environments

I think we should look at this and keep our options open. Plus, with all the
talent we have on our team, I feel this will be pretty easy to implement.

Post by Jeremy Jongsma
2) Proposing (and enforcing?) a Tiki API (particularly for module dev)

This is dearly needed. I can feel the potential for 3rd party modules &
plugins. In addition to making partnerships & recruitment a lot simpler,
here is a specific example of what a well defined API would solve:

Not all features are using all the transversal features. We should have a
standard list of transversal features which all basic modules.

For example: Trackers, Newsletter, Surveys, Charts and many more are not in
the search system. Trackers, Newsletter, Surveys, Charts and many others are
not in the category system. With talk of a category based permissions for
1.8, shouldn't we have a team with oversees the architecture?

Could the first step be to document a basic API?

How to implement a feature and it plays nicely with:
-User/group/permission system
-Search
-category system
-Templates
-rss feeds & communication center.
-modules
-WAP & PDF output
-admin menu

We have more people on the team. Let's make sure all the new people can
contribute.

+1 to a Team Architecture / team API / core dev / dev steering/ choose you
name ;-)

M ;-)

--
Marc Laporte

-----Original Message-----
From: tikiwiki-devel-***@lists.sourceforge.net
[mailto:tikiwiki-devel-***@lists.sourceforge.net] On Behalf Of Jeremy
Jongsma
Sent: August 2, 2003 2:06 PM
To: tikiwiki-***@lists.sourceforge.net
Subject: Re: [Tikiwiki-devel] TikiTeamArchitecture

Is anyone opposed to trying this incrementally? Starting with Michael's
suggestions would be a great start to make Tiki more modular and efficient.

Our team is growing fast, and I think it would be in everybody's best
interest to make these changes in order to allow the concept of "feature
teams" that, if desired, can work independently of others without
worrying too much about what other sections they're affecting, or if
another team is modifying the same files.

Not to mention that if we're going to be able to convince third parties
to develop their modules for Tiki, we need a well-defined feature API.
That's where an Architecture team would come in. Not to be a "ruling
cabal" of Tiki - but because if we let 60+ people add to the API as they
see fit, we end up like PostNuke - a bloated mess.

-j

http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01

Post by Gustavo Muslera
_______________________________________________
Tikiwiki-devel mailing list
https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel

--
Jeremy Jongsma
Co-Founder
TickChat
***@tickchat.com
http://www.tickchat.com

-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including
Data Reports, E-commerce, Portals, and Forums are available now.
Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01

Philippe

2003-08-05 12:04:50 UTC