Jump to content

Photo

Emerge rewrite in rust


13 replies to this topic

#1
gevious

gevious

    Newbie

  • Members
  • Pip
  • 7 posts
I'm relatively unfamiliar with the portage codebase, but know it's written in Python. I'm a web developer and use Python for writing server side code. Recently I have been intrigued by the rust programming language. I'm considering learning the language but need a good project to help me. I thought that maybe doing a rewrite of emerge would be a project worth considering since rust will be as secure as Python but will run significantly quicker.

I'd love to hear if this is something worth doing and if it would be able to help the community in the future.

#2
aramisqc

aramisqc

    Advanced Member

  • Members
  • PipPipPip
  • 36 posts
  • LocationSte-Anne-de-Beaupré QC (Canada)

That's something  very very big ! HUH !



#3
Oleg Vinichenko

Oleg Vinichenko

    Staff

  • Moderators
  • 380 posts

i'd prefer to clean-up obscure and undocumented python code in portage rather then rewrite it in new lang :)


  • 666threesixes666 and digifuzzy like this
www.funtoo.org

#4
aramisqc

aramisqc

    Advanced Member

  • Members
  • PipPipPip
  • 36 posts
  • LocationSte-Anne-de-Beaupré QC (Canada)

i'd prefer to clean-up obscure and undocumented python code in portage rather then rewrite it in new lang :)

 

+1



#5
spectromas

spectromas

    Advanced Member

  • Members
  • PipPipPip
  • 81 posts

I have no idea on the technical aspects but if emerge can be made quicker then it can only be a good thing as far as I'm concerned.



#6
digifuzzy

digifuzzy

    Advanced Member

  • Members
  • PipPipPip
  • 95 posts
  • LocationCanada

i'd prefer to clean-up obscure and undocumented python code in portage rather then rewrite it in new lang :)

+1

#7
aramisqc

aramisqc

    Advanced Member

  • Members
  • PipPipPip
  • 36 posts
  • LocationSte-Anne-de-Beaupré QC (Canada)

I have no idea on the technical aspects but if emerge can be made quicker then it can only be a good thing as far as I'm concerned.

 

Is Python the real reason for emerge being slow ? What is time consuming is when emerge calculates dependencies. I think so. I perhaps am wrong. If I were right, I would look to Portage tree data structure.

 

Python code could be written in C or C++. Both are close together. Would it be faster, quicker ? That's the question.



#8
Oleg Vinichenko

Oleg Vinichenko

    Staff

  • Moderators
  • 380 posts

there is C++ rewrite, called paludis and it's not any faster. A new portage version we testing locally is a bit faster. But last years, indeed, it slowed at some point. From what I understand portage now re-calculates it's dependency tree from scratch each time and depending on pkg list  it can be really discouraging. Maybe, we can make this behavior improved or changed completely. I.e problem is not because portage written in python, but rather in what it trying to do. (too smart?) 


www.funtoo.org

#9
aramisqc

aramisqc

    Advanced Member

  • Members
  • PipPipPip
  • 36 posts
  • LocationSte-Anne-de-Beaupré QC (Canada)

there is C++ rewrite, called paludis and it's not any faster. A new portage version we testing locally is a bit faster. But last years, indeed, it slowed at some point. From what I understand portage now re-calculates it's dependency tree from scratch each time and depending on pkg list  it can be really discouraging. Maybe, we can make this behavior improved or changed completely. I.e problem is not because portage written in python, but rather in what it trying to do. (too smart?) 

 

That's what I meant in my previous post. That's the way Portage calculates dependencies.



#10
gevious

gevious

    Newbie

  • Members
  • Pip
  • 7 posts
So, the speed improvements will come from better algorithms rather than faster code execution. Doing maintenance work on portage sounds like a good, albeit not-so-glamorous option. Thanks for the input

#11
Oleg Vinichenko

Oleg Vinichenko

    Staff

  • Moderators
  • 380 posts

sort of.  many blame portage for being slow and some such and complaining that python is the cause. yes python is not the fastest language but portage slowness is not caused by that. 


www.funtoo.org

#12
666threesixes666

666threesixes666

    Advanced Member

  • Members
  • PipPipPip
  • 129 posts

drobbins told me the speed of portage is limited by ebuilds calling bash when i was asking about rewrites in c.  (i thought paludis was c, oops :)  what are you guys doing, caching the dep tree?  im surprised the deps are calculated at all rather than defined within ebuilds?


Duke: No more of that talk or I'll put the f*cking leeches on you, understand?

 

Paul: [about Percy] The man is mean and careless and stupid, and that's a bad combination in a place like this.


#13
Oleg Vinichenko

Oleg Vinichenko

    Staff

  • Moderators
  • 380 posts

yes, bash is slow too, mainly because of readline library but still. The're several portage rewrites exist. paludis (C++), nix package manager (haskell), maybe others exist too. But they are mainly doing same as portage, i.e their existance dubious from point it's just portage (but in different lang). Devs who doesn't like python, created it without making anything revolutionary.  


www.funtoo.org

#14
digifuzzy

digifuzzy

    Advanced Member

  • Members
  • PipPipPip
  • 95 posts
  • LocationCanada

I would be leery of re-writing an important package management function in a language that (according to wikipedia) is at 1.0.0 Alpha stage.

 

I would be petrified after reading this on their front page

 

 

* In theory. Rust is a work-in-progress and may do anything it likes up to and including eating your laundry.

 

 


  • duncan.britton likes this



Reply to this topic



  


0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users